‘Might have a relevant impact on patient care, but to what extent does it align with the aims of this programme.’
Short title of aspects in the observation matrix . | Examples of arguments . |
---|---|
Criterion: scientific quality | |
Fit in programme objectives | ‘This disease is underdiagnosed, and undertreated, and therefore fits the criteria of this call very well.’ ‘Might have a relevant impact on patient care, but to what extent does it align with the aims of this programme.’ |
Match science and health-care problem | ‘It is not properly compared to the current situation (standard of care).’ ‘Super relevant application with a fitting plan, perhaps a little too mechanistic.’ |
International competitiveness | ‘Something is done all over the world, but they do many more evaluations, however.’ |
Feasibility of the aims | ‘… because this is a discovery study the power calculation is difficult, but I would recommend to increase the sample size.’ ‘It’s very risky, because this is an exploratory … study without hypotheses.’ ‘The aim is to improve …, but there is no control to compare with.’ ‘Well substantiated that they are able to achieve the objectives.’ |
Plan of work | ‘Will there be enough cases in this cohort?’ ‘The budget is no longer correct.’ ‘Plan is good, but … doubts about the approach, because too little information….’ |
Criterion: societal relevance | |
Health-care problem | ‘Relevant problem for a small group.’ ‘… but is this a serious health condition?’ ‘Prevalence is low, but patients do die, morbidity is very high.’ |
Contribution to solution | ‘What will this add since we already do…?’ ‘It is unclear what the intervention will be after the diagnosis.’ ‘Relevance is not good. Side effects are not known and neither is effectiveness.’ |
Next step in science | ‘What is needed to go from this retrospective study towards implementation?’ ‘It’s not clear whether that work package is necessary or “nice to have”.’ ‘Knowledge utilisation paragraph is standard, as used by copywriters.’ |
Activities towards partners | ‘What do the applicants do to change the current practice?’ ‘Important that the company also contributes financially to the further development.’ ‘This proposal includes a good communication plan.’ |
Participation/diversity | ‘A user committee is described, but it isn’t well thought through: what is their role?’ ‘It’s also important to invite relatives of patients to participate.’ ‘They thought really well what their patient group can contribute to the study plan.’ |
Applicant-related aspects | |
Scientific publication applicant | ‘One project leader only has one original paper, …, focus more on other diseases.’ ‘Publication output not excellent. Conference papers and posters of local meetings, CV not so strong.’ |
Background applicant | ‘… not enough with this expertise involved in the leadership.’ ‘Very good CV, … has won many awards.’ ‘Candidate is excellent, top 10 to 20 in this field….’ |
Reputation applicant | ‘… the main applicant is a hotshot in this field.’ ‘Candidate leads cohorts as …, gets a no.’ |
Societal skills | ‘Impressed that they took my question seriously, that made my day.’ ‘They were very honest about overoptimism in the proposal.’ ‘Good group, but they seem quite aware of their own brilliance.’ |
HTA | |
HTA | ‘Concrete revenues are negative, however improvement in quality-adjusted life years but very shaky.’ |
Committee-related aspects | |
Personal experience with the applicant | ‘This researcher only wants to acquire knowledge, nothing further.’ ‘I reviewed him before and he is not very good at interviews.’ |
Personal/unasserted preference | ‘Excellent presentation, much better than the application.’ (Without further elaboration) ‘This academic lab has advantages, but also disadvantages with regard to independence.’ ‘If it can be done anywhere, it is in this group.’ |
Relation with applicants’ institute/network | ‘May come up with new models, they’re linked with a group in … who can do this very well.’ |
Comparison with other applications | ‘What is the relevance compared to the other proposal? They do something similar.’ ‘Look at the proposals as a whole, portfolio, we have clinical and we have fundamental.’ |
Data were primarily collected through observations. Our observations of review panel meetings were non-participatory: the observer and goal of the observation were introduced at the start of the meeting, without further interactions during the meeting. To aid in the processing of observations, some meetings were audiotaped (sound only). Presentations or responses of applicants were not noted and were not part of the analysis. The observer made notes on the ongoing discussion and scored the arguments while listening. One meeting was not attended in person and only observed and scored by listening to the audiotape recording. Because this made identification of the panel members unreliable, this panel meeting was excluded from the analysis of the third research question on how arguments used differ between panel members with different perspectives.
We gathered and analysed all brochures and assessment forms used by the review panels in order to answer our second research question on the correspondence of arguments used with the formal criteria. Several programmes consisted of multiple grant calls: in that case, the specific call brochure was gathered and analysed, not the overall programme brochure. Additional documentation (e.g. instructional presentations at the start of the panel meeting) was not included in the document analysis. All included documents were marked using the aforementioned observation matrix. The panel-related arguments were not used because this category reflects the personal arguments of panel members that are not part of brochures or instructions. To avoid potential differences in scoring methods, two of the authors independently scored half of the documents that were checked and validated afterwards by the other. Differences were discussed until a consensus was reached.
In order to answer the third research question, background information on panel members was collected. We categorised the panel members into five common types of panel members: scientific, clinical scientific, health-care professional/clinical, patient, and policy. First, a list of all panel members was composed including their scientific and professional backgrounds and affiliations. The theoretical notion that reviewers represent different types of users of research and therefore potential impact domains (academic, social, economic, and cultural) was leading in the categorisation ( Meijer 2012 ; Spaapen and Van Drooge 2011 ). Because clinical researchers play a dual role in both advancing research as a fellow academic and as a user of the research output in health-care practice, we divided the academic members into two categories of non-clinical and clinical researchers. Multiple types of professional actors participated in each review panel. These were divided into two groups for the analysis: health-care professionals (without current academic activity) and policymakers in the health-care sector. No representatives of the private sector participated in the observed review panels. From the public domain, (at-risk) patients and patient representatives were part of several review panels. Only publicly available information was used to classify the panel members. Members were assigned to one category only: categorisation took place based on the specific role and expertise for which they were appointed to the panel.
In two of the four DHF programmes, the assessment procedure included the CSQ. In these two programmes, representatives of this CSQ participated in the scientific panel to articulate the findings of the CSQ meeting during the final assessment meeting. Two grant programmes were assessed by a review panel with solely (clinical) scientific members.
Data were processed using ATLAS.ti 8 and Microsoft Excel 2010 to produce descriptive statistics. All observed arguments were coded and given a randomised identification code for the panel member using that particular argument. The number of times an argument type was observed was used as an indicator for the relative importance of that argument in the appraisal of proposals. With this approach, a practical and reproducible method for research funders to evaluate the effect of policy changes on peer review was developed. If codes or notes were unclear, post-observation validation of codes was carried out based on observation matrix notes. Arguments that were noted by the observer but could not be matched with an existing code were first coded as a ‘non-existing’ code, and these were resolved by listening back to the audiotapes. Arguments that could not be assigned to a panel member were assigned a ‘missing panel member’ code. A total of 4.7 per cent of all codes were assigned a ‘missing panel member’ code.
After the analyses, two meetings were held to reflect on the results: one with the CSQ and the other with the programme coordinators of both organisations. The goal of these meetings was to improve our interpretation of the findings, disseminate the results derived from this project, and identify topics for further analyses or future studies.
Our study focuses on studying the final phase of the peer review process of research applications in a real-life setting. Our design, a non-participant observation of peer review panels, also introduced several challenges ( Liu and Maitlis 2010 ).
First, the independent review phase or pre-application phase was not part of our study. We therefore could not assess to what extent attention to certain aspects of scientific quality or societal relevance and impact in the review phase influenced the topics discussed during the meeting.
Second, the most important challenge of overt non-participant observations is the observer effect: the danger of causing reactivity in those under study. We believe that the consequences of this effect on our conclusions were limited because panellists are used to external observers in the meetings of these two funders. The observer briefly explained the goal of the study during the introductory round of the panel in general terms. The observer sat as unobtrusively as possible and avoided reactivity to discussions. Similar to previous observations of panels, we experienced that the fact that an observer was present faded into the background during a meeting ( Roumbanis 2021a ). However, a limited observer effect can never be entirely excluded.
Third, our design to only score the arguments raised, and not the responses of the applicant, or information on the content of the proposals, has its positives and negatives. With this approach, we could assure the anonymity of the grant procedures reviewed, the applicants and proposals, panels, and individual panellists. This was an important condition for the funders involved. We took the frequency arguments used as a proxy for the relative importance of that argument in decision-making, which undeniably also has its caveats. Our data collection approach limits more in-depth reflection on which arguments were decisive in decision-making and on group dynamics during the interaction with the applicants as non-verbal and non-content-related comments were not captured in this study.
Fourth, despite this being one of the largest observational studies on the peer review assessment of grant applications with the observation of ten panels in eight grant programmes, many variables might explain differences in arguments used within and beyond our view. Examples of ‘confounding’ variables are the many variations in panel composition, the differences in objectives of the programmes, and the range of the funding programmes. Our study should therefore be seen as exploratory and thus warrants caution in drawing conclusions.
The grant programmes included in this study reflected a broad range of biomedical and health funding programmes, ranging from fellowship grants to translational research and applied health research. All formal documents available to the applicants and to the review panel were retrieved for both ZonMw and the DHF. In total, eighteen documents corresponding to the eight grant programmes were studied. The number of proposals assessed per programme varied from three to thirty-three. The duration of the panel meetings varied between 2 h and two consecutive days. Together, this resulted in a large spread in the number of total arguments used in an individual meeting and in a grant programme as a whole. In the shortest meeting, 49 arguments were observed versus 254 in the longest, with a mean of 126 arguments per meeting and on average 15 arguments per proposal.
We found consistency between how criteria were operationalised in the grant programme’s brochures and in the assessment forms of the review panels overall. At the same time, because the number of elements included in the observation matrix is limited, there was a considerable diversity in the arguments that fall within each aspect (see examples in Table 1 ). Some of these differences could possibly be explained by differences in language used and the level of detail in the observation matrix, the brochure, and the panel’s instructions. This was especially the case in the applicant-related aspects in which the observation matrix was more detailed than the text in the brochure and assessment forms.
In interpretating our findings, it is important to take into account that, even though our data were largely complete and the observation matrix matched well with the description of the criteria in the brochures and assessment forms, there was a large diversity in the type and number of arguments used and in the number of proposals assessed in the grant programmes included in our study.
For our first research question, we explored the number and type of arguments used in the panel meetings. Figure 1 provides an overview of the arguments used. Scientific quality was discussed most. The number of times the feasibility of the aims was discussed clearly stands out in comparison to all other arguments. Also, the match between the science and the problem studied and the plan of work were frequently discussed aspects of scientific quality. International competitiveness of the proposal was discussed the least of all five scientific arguments.
The number of arguments used in panel meetings.
Attention was paid to societal relevance and impact in the panel meetings of both organisations. Yet, the language used differed somewhat between organisations. The contribution to a solution and the next step in science were the most often used societal arguments. At ZonMw, the impact of the health-care problem studied and the activities towards partners were less frequently discussed than the other three societal arguments. At the DHF, the five societal arguments were used equally often.
With the exception of the fellowship programme meeting, applicant-related arguments were not often used. The fellowship panel used arguments related to the applicant and to scientific quality about equally often. Committee-related arguments were also rarely used in the majority of the eight grant programmes observed. In three out of the ten panel meetings, one or two arguments were observed, which were related to personal experience with the applicant or their direct network. In seven out of ten meetings, statements were observed, which were unasserted or were explicitly announced as reflecting a personal preference. The frequency varied between one and seven statements (sixteen in total), which is low in comparison to the other arguments used (see Fig. 1 for examples).
The balance in the use of scientific and societal arguments varied strongly per grant programme, panel, and organisation. At ZonMw, two meetings had approximately an equal balance in societal and scientific arguments. In the other two meetings, scientific arguments were used twice to four times as often as societal arguments. At the DHF, three types of panels were observed. Different patterns in the relative use of societal and scientific arguments were observed for each of these panel types. In the two CSQ-only meetings the societal arguments were used approximately twice as often as scientific arguments. In the two meetings of the scientific panels, societal arguments were infrequently used (between zero and four times per argument category). In the combined societal and scientific panel meetings, the use of societal and scientific arguments was more balanced.
In order to answer our second research question, we looked into the relation of the arguments used with the formal criteria. We observed that a broader range of arguments were often used in comparison to how the criteria were described in the brochure and assessment instruction. However, arguments related to aspects that were consequently included in the brochure and instruction seemed to be discussed more frequently than in programmes where those aspects were not consistently included or were not included at all. Although the match of the science with the health-care problem and the background and reputation of the applicant were not always made explicit in the brochure or instructions, they were discussed in many panel meetings. Supplementary Fig. S1 provides a visualisation of how arguments used differ between the programmes in which those aspects were, were not, consistently included in the brochure and instruction forms.
To answer our third question, we looked into the differences in arguments used between panel members representing a scientific, clinical scientific, professional, policy, or patient perspective. In each research programme, the majority of panellists had a scientific background ( n = 35), thirty-four members had a clinical scientific background, twenty had a health professional/clinical background, eight members represented a policy perspective, and fifteen represented a patient perspective. From the total number of arguments (1,097), two-thirds were made by members with a scientific or clinical scientific perspective. Members with a scientific background engaged most actively in the discussion with a mean of twelve arguments per member. Similarly, clinical scientists and health-care professionals participated with a mean of nine arguments, and members with a policy and patient perspective put forward the least number of arguments on average, namely, seven and eight. Figure 2 provides a complete overview of the total and mean number of arguments used by the different disciplines in the various panels.
The total and mean number of arguments displayed per subgroup of panel members.
In meetings of both organisations, we observed a diverse use of arguments by the panel members. Yet, the use of arguments varied depending on the background of the panel member (see Fig. 3 ). Those with a scientific and clinical scientific perspective used primarily scientific arguments. As could be expected, health-care professionals and patients used societal arguments more often.
The use of arguments differentiated by panel member background.
Further breakdown of arguments across backgrounds showed clear differences in the use of scientific arguments between the different disciplines of panellists. Scientists and clinical scientists discussed the feasibility of the aims more than twice as often as their second most often uttered element of scientific quality, which was the match between the science and the problem studied . Patients and members with a policy or health professional background put forward fewer but more varied scientific arguments.
Patients and health-care professionals accounted for approximately half of the societal arguments used, despite being a much smaller part of the panel’s overall composition. In other words, members with a scientific perspective were less likely to use societal arguments. The relevance of the health-care problem studied, activities towards partners , and arguments related to participation and diversity were not used often by this group. Patients often used arguments related to patient participation and diversity and activities towards partners , although the frequency of the use of the latter differed per organisation.
The majority of the applicant-related arguments were put forward by scientists, including clinical scientists. Committee-related arguments were very rare and are therefore not differentiated by panel member background, except comments related to a comparison with other applications. These arguments were mainly put forward by panel members with a scientific background. HTA -related arguments were often used by panel members with a scientific perspective. Panel members with other perspectives used this argument scarcely (see Supplementary Figs S2–S4 for the visual presentation of the differences between panel members on all aspects included in the matrix).
Our observations show that most arguments for scientific quality were often used. However, except for the feasibility , the frequency of arguments used varied strongly between the meetings and between the individual proposals that were discussed. The fact that most arguments were not consistently used is not surprising given the results from previous studies that showed heterogeneity in grant application assessments and low consistency in comments and scores by independent reviewers ( Abdoul et al. 2012 ; Pier et al. 2018 ). In an analysis of written assessments on nine observed dimensions, no dimension was used in more than 45 per cent of the reviews ( Hartmann and Neidhardt 1990 ).
There are several possible explanations for this heterogeneity. Roumbanis (2021a) described how being responsive to the different challenges in the proposals and to the points of attention arising from the written assessments influenced discussion in panels. Also when a disagreement arises, more time is spent on discussion ( Roumbanis 2021a ). One could infer that unambiguous, and thus not debated, aspects might remain largely undetected in our study. We believe, however, that the main points relevant to the assessment will not remain entirely unmentioned, because most panels in our study started the discussion with a short summary of the proposal, the written assessment, and the rebuttal. Lamont (2009) , however, points out that opening statements serve more goals than merely decision-making. They can also increase the credibility of the panellist, showing their comprehension and balanced assessment of an application. We can therefore not entirely disentangle whether the arguments observed most were also found to be most important or decisive or those were simply the topics that led to most disagreement.
An interesting difference with Roumbanis’ study was the available discussion time per proposal. In our study, most panels handled a limited number of proposals, allowing for longer discussions in comparison with the often 2-min time frame that Roumbanis (2021b) described, potentially contributing to a wider range of arguments being discussed. Limited time per proposal might also limit the number of panellists contributing to the discussion per proposal ( De Bont 2014 ).
We found that the language used for the operationalisation of the assessment criteria in programme brochures and in the observation matrix was much more detailed than in the instruction for the panel, which was often very concise. The exercise also illustrated that many terms were used interchangeably.
This was especially true for the applicant-related aspects. Several panels discussed how talent should be assessed. This confusion is understandable when considering the changing values in research and its assessment ( Moher et al. 2018 ) and the fact that the instruction of the funders was very concise. For example, it was not explicated whether the individual or the team should be assessed. Arensbergen et al. (2014b) described how in grant allocation processes, talent is generally assessed using limited characteristics. More objective and quantifiable outputs often prevailed at the expense of recognising and rewarding a broad variety of skills and traits combining professional, social, and individual capital ( DORA 2013 ).
In addition, committee-related arguments, like personal experiences with the applicant or their institute, were rarely used in our study. Comparisons between proposals were sometimes made without further argumentation, mainly by scientific panel members. This was especially pronounced in one (fellowship) grant programme with a high number of proposals. In this programme, the panel meeting concentrated on quickly comparing the quality of the applicants and of the proposals based on the reviewer’s judgement, instead of a more in-depth discussion of the different aspects of the proposals. Because the review phase was not part of this study, the question of which aspects have been used for the assessment of the proposals in this panel therefore remains partially unanswered. However, weighing and comparing proposals on different aspects and with different inputs is a core element of scientific peer review, both in the review of papers and in the review of grants ( Hirschauer 2010 ). The large role of scientific panel members in comparing proposals is therefore not surprising.
One could anticipate that more consequent language in the operationalising criteria may lead to more clarity for both applicants and panellists and to more consistency in the assessment of research proposals. The trend in our observations was that arguments were used less when the related criteria were not or were consequently included in the brochure and panel instruction. It remains, however, challenging to disentangle the influence of the formal definitions of criteria on the arguments used. Previous studies also encountered difficulties in studying the role of the formal instruction in peer review but concluded that this role is relatively limited ( Langfeldt 2001 ; Reinhart 2010 ).
The lack of a clear operationalisation of criteria can contribute to heterogeneity in peer review as many scholars found that assessors differ in the conceptualisation of good science and to the importance they attach to various aspects of research quality and societal relevance ( Abdoul et al. 2012 ; Geurts 2016 ; Scholten et al. 2018 ; Van den Brink et al. 2016 ). The large variation and absence of a gold standard in the interpretation of scientific quality and societal relevance affect the consistency of peer review. As a consequence, it is challenging to systematically evaluate and improve peer review in order to fund the research that contributes most to science and society. To contribute to responsible research and innovation, it is, therefore, important that funders invest in a more consistent and conscientious peer review process ( Curry et al. 2020 ; DORA 2013 ).
A common conceptualisation of scientific quality and societal relevance and impact could improve the alignment between views on good scientific conduct, programmes’ objectives, and the peer review in practice. Such a conceptualisation could contribute to more transparency and quality in the assessment of research. By involving panel members from all relevant backgrounds, including the research community, health-care professionals, and societal actors, in a better operationalisation of criteria, more inclusive views of good science can be implemented more systematically in the peer review assessment of research proposals. The ZonMw Framework Fostering Responsible Research Practices is an example of an initiative aiming to support standardisation and integration ( Reijmerink et al. 2020 ).
Given the lack of a common definition or conceptualisation of scientific quality and societal relevance, our study made an important decision by choosing to use a fixed set of detailed aspects of two important criteria as a gold standard to score the brochures, the panel instructions, and the arguments used by the panels. This approach proved helpful in disentangling the different components of scientific quality and societal relevance. Having said that, it is important not to oversimplify the causes for heterogeneity in peer review because these substantive arguments are not independent of non-cognitive, emotional, or social aspects ( Lamont and Guetzkow 2016 ; Reinhart 2010 ).
Both funders participating in our study have an outspoken public mission that requests sufficient attention to societal aspects in assessment processes. In reality, as observed in several panels, the main focus of peer review meetings is on scientific arguments. Next to the possible explanations earlier, the composition of the panel might play a role in explaining arguments used in panel meetings. Our results have shown that health-care professionals and patients bring in more societal arguments than scientists, including those who are also clinicians. It is, however, not that simple. In the more diverse panels, panel members, regardless of their backgrounds, used more societal arguments than in the less diverse panels.
Observing ten panel meetings was sufficient to explore differences in arguments used by panel members with different backgrounds. The pattern of (primarily) scientific arguments being raised by panels with mainly scientific members is not surprising. After all, it is their main task to assess the scientific content of grant proposals and fit their competencies. As such, one could argue, depending on how one justifies the relationship between science and society, that health-care professionals and patients might be better suited to assess the value for potential users of research results. Scientific panel members and clinical scientists in our study used less arguments that reflect on opening up and connecting science directly to others who can bring it further (being industry, health-care professionals, or other stakeholders). Patients filled this gap since these two types of arguments were the most prevalent type put forward by them. Making an active connection with society apparently needs a broader, more diverse panel for scientists to direct their attention to more societal arguments. Evident from our observations is that in panels with patients and health-care professionals, their presence seemed to increase the attention placed on arguments beyond the scientific arguments put forward by all panel members, including scientists. This conclusion is congruent with the observation that there was a more equal balance in the use of societal and scientific arguments in the scientific panels in which the CSQ participated. This illustrates that opening up peer review panels to non-scientific members creates an opportunity to focus on both the contribution and the integrative rationality ( Glerup and Horst 2014 ) or, in other words, to allow productive interactions between scientific and non-scientific actors. This corresponds with previous research that suggests that with regard to societal aspects, reviews from mixed panels were broader and richer ( Luo et al. 2021 ). In panels with non-scientific experts, more emphasis was placed on the role of the proposed research process to increase the likelihood of societal impact over the causal importance of scientific excellence for broader impacts. This is in line with the findings that panels with more disciplinary diversity, in range and also by including generalist experts, applied more versatile styles to reach consensus and paid more attention to relevance and pragmatic value ( Huutoniemi 2012 ).
Our observations further illustrate that patients and health-care professionals were less vocal in panels than (clinical) scientists and were in the minority. This could reflect their social role and lower perceived authority in the panel. Several guides are available for funders to stimulate the equal participation of patients in science. These guides are also applicable to their involvement in peer review panels. Measures to be taken include the support and training to help prepare patients for their participation in deliberations with renowned scientists and explicitly addressing power differences ( De Wit et al. 2016 ). Panel chairs and programme officers have to set and supervise the conditions for the functioning of both the individual panel members and the panel as a whole ( Lamont 2009 ).
In future studies, it is important to further disentangle the role of the operationalisation and appraisal of assessment criteria in reducing heterogeneity in the arguments used by panels. More controlled experimental settings are a valuable addition to the current mainly observational methodologies applied to disentangle some of the cognitive and social factors that influence the functioning and argumentation of peer review panels. Reusing data from the panel observations and the data on the written reports could also provide a starting point for a bottom-up approach to create a more consistent and shared conceptualisation and operationalisation of assessment criteria.
To further understand the effects of opening up review panels to non-scientific peers, it is valuable to compare the role of diversity and interdisciplinarity in solely scientific panels versus panels that also include non-scientific experts.
In future studies, differences between domains and types of research should also be addressed. We hypothesise that biomedical and health research is perhaps more suited for the inclusion of non-scientific peers in panels than other research domains. For example, it is valuable to better understand how potentially relevant users can be well enough identified in other research fields and to what extent non-academics can contribute to assessing the possible value of, especially early or blue sky, research.
The goal of our study was to explore in practice which arguments regarding the main criteria of scientific quality and societal relevance were used by peer review panels of biomedical and health research funding programmes. We showed that there is a wide diversity in the number and range of arguments used, but three main scientific aspects were discussed most frequently. These are the following: is it a feasible approach; does the science match the problem , and is the work plan scientifically sound? Nevertheless, these scientific aspects were accompanied by a significant amount of discussion of societal aspects, of which the contribution to a solution is the most prominent. In comparison with scientific panellists, non-scientific panellists, such as health-care professionals, policymakers, and patients, often use a wider range of arguments and other societal arguments. Even more striking was that, even though non-scientific peers were often outnumbered and less vocal in panels, scientists also used a wider range of arguments when non-scientific peers were present.
It is relevant that two health research funders collaborated in the current study to reflect on and improve peer review in research funding. There are few studies published that describe live observations of peer review panel meetings. Many studies focus on alternatives for peer review or reflect on the outcomes of the peer review process, instead of reflecting on the practice and improvement of peer review assessment of grant proposals. Privacy and confidentiality concerns of funders also contribute to the lack of information on the functioning of peer review panels. In this study, both organisations were willing to participate because of their interest in research funding policies in relation to enhancing the societal value and impact of science. The study provided them with practical suggestions, for example, on how to improve the alignment in language used in programme brochures and instructions of review panels, and contributed to valuable knowledge exchanges between organisations. We hope that this publication stimulates more research funders to evaluate their peer review approach in research funding and share their insights.
For a long time, research funders relied solely on scientists for designing and executing peer review of research proposals, thereby delegating responsibility for the process. Although review panels have a discretionary authority, it is important that funders set and supervise the process and the conditions. We argue that one of these conditions should be the diversification of peer review panels and opening up panels for non-scientific peers.
Supplementary material is available at Science and Public Policy online.
Details of the data and information on how to request access is available from the first author.
Joey Gijbels and Wendy Reijmerink are employed by ZonMw. Rebecca Abma-Schouten is employed by the Dutch Heart Foundation and as external PhD candidate affiliated with the Centre for Science and Technology Studies, Leiden University.
A special thanks to the panel chairs and programme officers of ZonMw and the DHF for their willingness to participate in this project. We thank Diny Stekelenburg, an internship student at ZonMw, for her contributions to the project. Our sincerest gratitude to Prof. Paul Wouters, Sarah Coombs, and Michiel van der Vaart for proofreading and their valuable feedback. Finally, we thank the editors and anonymous reviewers of Science and Public Policy for their thorough and insightful reviews and recommendations. Their contributions are recognisable in the final version of this paper.
Abdoul H. , Perrey C. , Amiel P. , et al. ( 2012 ) ‘ Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices ’, PLoS One , 7 : 1 – 15 .
Google Scholar
Abma-Schouten R. Y. ( 2017 ) ‘ Maatschappelijke Kwaliteit van Onderzoeksvoorstellen ’, Dutch Heart Foundation .
Alla K. , Hall W. D. , Whiteford H. A. , et al. ( 2017 ) ‘ How Do We Define the Policy Impact of Public Health Research? A Systematic Review ’, Health Research Policy and Systems , 15 : 84.
Benedictus R. , Miedema F. , and Ferguson M. W. J. ( 2016 ) ‘ Fewer Numbers, Better Science ’, Nature , 538 : 453 – 4 .
Chalmers I. , Bracken M. B. , Djulbegovic B. , et al. ( 2014 ) ‘ How to Increase Value and Reduce Waste When Research Priorities Are Set ’, The Lancet , 383 : 156 – 65 .
Curry S. , De Rijcke S. , Hatch A. , et al. ( 2020 ) ‘ The Changing Role of Funders in Responsible Research Assessment: Progress, Obstacles and the Way Ahead ’, RoRI Working Paper No. 3, London : Research on Research Institute (RoRI) .
De Bont A. ( 2014 ) ‘ Beoordelen Bekeken. Reflecties op het Werk van Een Programmacommissie van ZonMw ’, ZonMw .
De Rijcke S. , Wouters P. F. , Rushforth A. D. , et al. ( 2016 ) ‘ Evaluation Practices and Effects of Indicator Use—a Literature Review ’, Research Evaluation , 25 : 161 – 9 .
De Wit A. M. , Bloemkolk D. , Teunissen T. , et al. ( 2016 ) ‘ Voorwaarden voor Succesvolle Betrokkenheid van Patiënten/cliënten bij Medisch Wetenschappelijk Onderzoek ’, Tijdschrift voor Sociale Gezondheidszorg , 94 : 91 – 100 .
Del Carmen Calatrava Moreno M. , Warta K. , Arnold E. , et al. ( 2019 ) Science Europe Study on Research Assessment Practices . Technopolis Group Austria .
Google Preview
Demicheli V. and Di Pietrantonj C. ( 2007 ) ‘ Peer Review for Improving the Quality of Grant Applications ’, Cochrane Database of Systematic Reviews , 2 : MR000003.
Den Oudendammer W. M. , Noordhoek J. , Abma-Schouten R. Y. , et al. ( 2019 ) ‘ Patient Participation in Research Funding: An Overview of When, Why and How Amongst Dutch Health Funds ’, Research Involvement and Engagement , 5 .
Diabetesfonds ( n.d. ) Maatschappelijke Adviesraad < https://www.diabetesfonds.nl/over-ons/maatschappelijke-adviesraad > accessed 18 Sept 2022 .
Dijstelbloem H. , Huisman F. , Miedema F. , et al. ( 2013 ) ‘ Science in Transition Position Paper: Waarom de Wetenschap Niet Werkt Zoals het Moet, En Wat Daar aan te Doen Is ’, Utrecht : Science in Transition .
Forsyth D. R. ( 1999 ) Group Dynamics , 3rd edn. Belmont : Wadsworth Publishing Company .
Geurts J. ( 2016 ) ‘ Wat Goed Is, Herken Je Meteen ’, NRC Handelsblad < https://www.nrc.nl/nieuws/2016/10/28/wat-goed-is-herken-je-meteen-4975248-a1529050 > accessed 6 Mar 2022 .
Glerup C. and Horst M. ( 2014 ) ‘ Mapping “Social Responsibility” in Science ’, Journal of Responsible Innovation , 1 : 31 – 50 .
Hartmann I. and Neidhardt F. ( 1990 ) ‘ Peer Review at the Deutsche Forschungsgemeinschaft ’, Scientometrics , 19 : 419 – 25 .
Hirschauer S. ( 2010 ) ‘ Editorial Judgments: A Praxeology of “Voting” in Peer Review ’, Social Studies of Science , 40 : 71 – 103 .
Hughes A. and Kitson M. ( 2012 ) ‘ Pathways to Impact and the Strategic Role of Universities: New Evidence on the Breadth and Depth of University Knowledge Exchange in the UK and the Factors Constraining Its Development ’, Cambridge Journal of Economics , 36 : 723 – 50 .
Huutoniemi K. ( 2012 ) ‘ Communicating and Compromising on Disciplinary Expertise in the Peer Review of Research Proposals ’, Social Studies of Science , 42 : 897 – 921 .
Jasanoff S. ( 2011 ) ‘ Constitutional Moments in Governing Science and Technology ’, Science and Engineering Ethics , 17 : 621 – 38 .
Kolarz P. , Arnold E. , Farla K. , et al. ( 2016 ) Evaluation of the ESRC Transformative Research Scheme . Brighton : Technopolis Group .
Lamont M. ( 2009 ) How Professors Think : Inside the Curious World of Academic Judgment . Cambridge : Harvard University Press .
Lamont M. Guetzkow J. ( 2016 ) ‘How Quality Is Recognized by Peer Review Panels: The Case of the Humanities’, in M. Ochsner , S. E. Hug , and H.-D. Daniel (eds) Research Assessment in the Humanities , pp. 31 – 41 . Cham : Springer International Publishing .
Lamont M. Huutoniemi K. ( 2011 ) ‘Comparing Customary Rules of Fairness: Evaluative Practices in Various Types of Peer Review Panels’, in C. Charles G. Neil and L. Michèle (eds) Social Knowledge in the Making , pp. 209–32. Chicago : The University of Chicago Press .
Langfeldt L. ( 2001 ) ‘ The Decision-making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome ’, Social Studies of Science , 31 : 820 – 41 .
——— ( 2006 ) ‘ The Policy Challenges of Peer Review: Managing Bias, Conflict of Interests and Interdisciplinary Assessments ’, Research Evaluation , 15 : 31 – 41 .
Lee C. J. , Sugimoto C. R. , Zhang G. , et al. ( 2013 ) ‘ Bias in Peer Review ’, Journal of the American Society for Information Science and Technology , 64 : 2 – 17 .
Liu F. Maitlis S. ( 2010 ) ‘Nonparticipant Observation’, in A. J. Mills , G. Durepos , and E. Wiebe (eds) Encyclopedia of Case Study Research , pp. 609 – 11 . Los Angeles : SAGE .
Luo J. , Ma L. , and Shankar K. ( 2021 ) ‘ Does the Inclusion of Non-academix Reviewers Make Any Difference for Grant Impact Panels? ’, Science & Public Policy , 48 : 763 – 75 .
Luukkonen T. ( 2012 ) ‘ Conservatism and Risk-taking in Peer Review: Emerging ERC Practices ’, Research Evaluation , 21 : 48 – 60 .
Macleod M. R. , Michie S. , Roberts I. , et al. ( 2014 ) ‘ Biomedical Research: Increasing Value, Reducing Waste ’, The Lancet , 383 : 101 – 4 .
Meijer I. M. ( 2012 ) ‘ Societal Returns of Scientific Research. How Can We Measure It? ’, Leiden : Center for Science and Technology Studies, Leiden University .
Merton R. K. ( 1968 ) Social Theory and Social Structure , Enlarged edn. [Nachdr.] . New York : The Free Press .
Moher D. , Naudet F. , Cristea I. A. , et al. ( 2018 ) ‘ Assessing Scientists for Hiring, Promotion, And Tenure ’, PLoS Biology , 16 : e2004089.
Olbrecht M. and Bornmann L. ( 2010 ) ‘ Panel Peer Review of Grant Applications: What Do We Know from Research in Social Psychology on Judgment and Decision-making in Groups? ’, Research Evaluation , 19 : 293 – 304 .
Patiëntenfederatie Nederland ( n.d. ) Ervaringsdeskundigen Referentenpanel < https://www.patientenfederatie.nl/zet-je-ervaring-in/lid-worden-van-ons-referentenpanel > accessed 18 Sept 2022.
Pier E. L. , M. B. , Filut A. , et al. ( 2018 ) ‘ Low Agreement among Reviewers Evaluating the Same NIH Grant Applications ’, Proceedings of the National Academy of Sciences , 115 : 2952 – 7 .
Prinses Beatrix Spierfonds ( n.d. ) Gebruikerscommissie < https://www.spierfonds.nl/wie-wij-zijn/gebruikerscommissie > accessed 18 Sep 2022 .
( 2020 ) Private Non-profit Financiering van Onderzoek in Nederland < https://www.rathenau.nl/nl/wetenschap-cijfers/geld/wat-geeft-nederland-uit-aan-rd/private-non-profit-financiering-van#:∼:text=R%26D%20in%20Nederland%20wordt%20gefinancierd,aan%20wetenschappelijk%20onderzoek%20in%20Nederland > accessed 6 Mar 2022 .
Reneman R. S. , Breimer M. L. , Simoons J. , et al. ( 2010 ) ‘ De toekomst van het cardiovasculaire onderzoek in Nederland. Sturing op synergie en impact ’, Den Haag : Nederlandse Hartstichting .
Reed M. S. , Ferré M. , Marin-Ortega J. , et al. ( 2021 ) ‘ Evaluating Impact from Research: A Methodological Framework ’, Research Policy , 50 : 104147.
Reijmerink W. and Oortwijn W. ( 2017 ) ‘ Bevorderen van Verantwoorde Onderzoekspraktijken Door ZonMw ’, Beleidsonderzoek Online. accessed 6 Mar 2022.
Reijmerink W. , Vianen G. , Bink M. , et al. ( 2020 ) ‘ Ensuring Value in Health Research by Funders’ Implementation of EQUATOR Reporting Guidelines: The Case of ZonMw ’, Berlin : REWARD|EQUATOR .
Reinhart M. ( 2010 ) ‘ Peer Review Practices: A Content Analysis of External Reviews in Science Funding ’, Research Evaluation , 19 : 317 – 31 .
Reinhart M. and Schendzielorz C. ( 2021 ) Trends in Peer Review . SocArXiv . < https://osf.io/preprints/socarxiv/nzsp5 > accessed 29 Aug 2022.
Roumbanis L. ( 2017 ) ‘ Academic Judgments under Uncertainty: A Study of Collective Anchoring Effects in Swedish Research Council Panel Groups ’, Social Studies of Science , 47 : 95 – 116 .
——— ( 2021a ) ‘ Disagreement and Agonistic Chance in Peer Review ’, Science, Technology & Human Values , 47 : 1302 – 33 .
——— ( 2021b ) ‘ The Oracles of Science: On Grant Peer Review and Competitive Funding ’, Social Science Information , 60 : 356 – 62 .
( 2019 ) ‘ Ruimte voor ieders talent (Position Paper) ’, Den Haag : VSNU, NFU, KNAW, NWO en ZonMw . < https://www.universiteitenvannederland.nl/recognitionandrewards/wp-content/uploads/2019/11/Position-paper-Ruimte-voor-ieders-talent.pdf >.
( 2013 ) San Francisco Declaration on Research Assessment . The Declaration . < https://sfdora.org > accessed 2 Jan 2022 .
Sarewitz D. and Pielke R. A. Jr. ( 2007 ) ‘ The Neglected Heart of Science Policy: Reconciling Supply of and Demand for Science ’, Environmental Science & Policy , 10 : 5 – 16 .
Scholten W. , Van Drooge L. , and Diederen P. ( 2018 ) Excellent Is Niet Gewoon. Dertig Jaar Focus op Excellentie in het Nederlandse Wetenschapsbeleid . The Hague : Rathenau Instituut .
Shapin S. ( 2008 ) The Scientific Life : A Moral History of a Late Modern Vocation . Chicago : University of Chicago press .
Spaapen J. and Van Drooge L. ( 2011 ) ‘ Introducing “Productive Interactions” in Social Impact Assessment ’, Research Evaluation , 20 : 211 – 8 .
Travis G. D. L. and Collins H. M. ( 1991 ) ‘ New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System ’, Science, Technology & Human Values , 16 : 322 – 41 .
Van Arensbergen P. and Van den Besselaar P. ( 2012 ) ‘ The Selection of Scientific Talent in the Allocation of Research Grants ’, Higher Education Policy , 25 : 381 – 405 .
Van Arensbergen P. , Van der Weijden I. , and Van den Besselaar P. V. D. ( 2014a ) ‘ The Selection of Talent as a Group Process: A Literature Review on the Social Dynamics of Decision Making in Grant Panels ’, Research Evaluation , 23 : 298 – 311 .
—— ( 2014b ) ‘ Different Views on Scholarly Talent: What Are the Talents We Are Looking for in Science? ’, Research Evaluation , 23 : 273 – 84 .
Van den Brink , G. , Scholten , W. , and Jansen , T. , eds ( 2016 ) Goed Werk voor Academici . Culemborg : Stichting Beroepseer .
Weingart P. ( 1999 ) ‘ Scientific Expertise and Political Accountability: Paradoxes of Science in Politics ’, Science & Public Policy , 26 : 151 – 61 .
Wessely S. ( 1998 ) ‘ Peer Review of Grant Applications: What Do We Know? ’, The Lancet , 352 : 301 – 5 .
Month: | Total Views: |
---|---|
April 2023 | 723 |
May 2023 | 266 |
June 2023 | 152 |
July 2023 | 130 |
August 2023 | 355 |
September 2023 | 189 |
October 2023 | 198 |
November 2023 | 181 |
December 2023 | 153 |
January 2024 | 197 |
February 2024 | 222 |
March 2024 | 227 |
April 2024 | 218 |
May 2024 | 229 |
June 2024 | 81 |
Citing articles via.
Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide
Sign In or Create an Account
This PDF is available to Subscribers Only
For full access to this pdf, sign in to an existing account, or purchase an annual subscription.
Run a free plagiarism check in 10 minutes, generate accurate citations for free.
Published on October 12, 2022 by Shona McCombes and Tegan George. Revised on November 21, 2023.
A research proposal describes what you will investigate, why it’s important, and how you will conduct your research.
The format of a research proposal varies between fields, but most proposals will contain at least these elements:
Literature review.
While the sections may vary, the overall objective is always the same. A research proposal serves as a blueprint and guide for your research plan, helping you get organized and feel confident in the path forward you choose to take.
Research proposal purpose, research proposal examples, research design and methods, contribution to knowledge, research schedule, other interesting articles, frequently asked questions about research proposals.
Academics often have to write research proposals to get funding for their projects. As a student, you might have to write a research proposal as part of a grad school application , or prior to starting your thesis or dissertation .
In addition to helping you figure out what your research can look like, a proposal can also serve to demonstrate why your project is worth pursuing to a funder, educational institution, or supervisor.
Show your reader why your project is interesting, original, and important. | |
Demonstrate your comfort and familiarity with your field. Show that you understand the current state of research on your topic. | |
Make a case for your . Demonstrate that you have carefully thought about the data, tools, and procedures necessary to conduct your research. | |
Confirm that your project is feasible within the timeline of your program or funding deadline. |
The length of a research proposal can vary quite a bit. A bachelor’s or master’s thesis proposal can be just a few pages, while proposals for PhD dissertations or research funding are usually much longer and more detailed. Your supervisor can help you determine the best length for your work.
One trick to get started is to think of your proposal’s structure as a shorter version of your thesis or dissertation , only without the results , conclusion and discussion sections.
Download our research proposal template
Discover proofreading & editing
Writing a research proposal can be quite challenging, but a good starting point could be to look at some examples. We’ve included a few for you below.
Like your dissertation or thesis, the proposal will usually have a title page that includes:
The first part of your proposal is the initial pitch for your project. Make sure it succinctly explains what you want to do and why.
Your introduction should:
To guide your introduction , include information about:
As you get started, it’s important to demonstrate that you’re familiar with the most important research on your topic. A strong literature review shows your reader that your project has a solid foundation in existing knowledge or theory. It also shows that you’re not simply repeating what other people have already done or said, but rather using existing research as a jumping-off point for your own.
In this section, share exactly how your project will contribute to ongoing conversations in the field by:
Following the literature review, restate your main objectives . This brings the focus back to your own project. Next, your research design or methodology section will describe your overall approach, and the practical steps you will take to answer your research questions.
? or ? , , or research design? | |
, )? ? | |
, , , )? | |
? |
To finish your proposal on a strong note, explore the potential implications of your research for your field. Emphasize again what you aim to contribute and why it matters.
For example, your results might have implications for:
Last but not least, your research proposal must include correct citations for every source you have used, compiled in a reference list . To create citations quickly and easily, you can use our free APA citation generator .
Some institutions or funders require a detailed timeline of the project, asking you to forecast what you will do at each stage and how long it may take. While not always required, be sure to check the requirements of your project.
Here’s an example schedule to help you get started. You can also download a template at the button below.
Download our research schedule template
Research phase | Objectives | Deadline |
---|---|---|
1. Background research and literature review | 20th January | |
2. Research design planning | and data analysis methods | 13th February |
3. Data collection and preparation | with selected participants and code interviews | 24th March |
4. Data analysis | of interview transcripts | 22nd April |
5. Writing | 17th June | |
6. Revision | final work | 28th July |
If you are applying for research funding, chances are you will have to include a detailed budget. This shows your estimates of how much each part of your project will cost.
Make sure to check what type of costs the funding body will agree to cover. For each item, include:
To determine your budget, think about:
If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.
Methodology
Statistics
Research bias
Once you’ve decided on your research objectives , you need to explain them in your paper, at the end of your problem statement .
Keep your research objectives clear and concise, and use appropriate verbs to accurately convey the work that you will carry out for each one.
I will compare …
A research aim is a broad statement indicating the general purpose of your research project. It should appear in your introduction at the end of your problem statement , before your research objectives.
Research objectives are more specific than your research aim. They indicate the specific ways you’ll address the overarching aim.
A PhD, which is short for philosophiae doctor (doctor of philosophy in Latin), is the highest university degree that can be obtained. In a PhD, students spend 3–5 years writing a dissertation , which aims to make a significant, original contribution to current knowledge.
A PhD is intended to prepare students for a career as a researcher, whether that be in academia, the public sector, or the private sector.
A master’s is a 1- or 2-year graduate degree that can prepare you for a variety of careers.
All master’s involve graduate-level coursework. Some are research-intensive and intend to prepare students for further study in a PhD; these usually require their students to write a master’s thesis . Others focus on professional training for a specific career.
Critical thinking refers to the ability to evaluate information and to be aware of biases or assumptions, including your own.
Like information literacy , it involves evaluating arguments, identifying and solving problems in an objective and systematic way, and clearly communicating your ideas.
The best way to remember the difference between a research plan and a research proposal is that they have fundamentally different audiences. A research plan helps you, the researcher, organize your thoughts. On the other hand, a dissertation proposal or research proposal aims to convince others (e.g., a supervisor, a funding body, or a dissertation committee) that your research topic is relevant and worthy of being conducted.
If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.
McCombes, S. & George, T. (2023, November 21). How to Write a Research Proposal | Examples & Templates. Scribbr. Retrieved June 18, 2024, from https://www.scribbr.com/research-process/research-proposal/
Other students also liked, how to write a problem statement | guide & examples, writing strong research questions | criteria & examples, how to write a literature review | guide, examples, & templates, get unlimited documents corrected.
✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts
941 Accesses
A research proposal is a roadmap that brings the researcher closer to the objectives, takes the research topic from a purely subjective mind, and manifests an objective plan. It shows us what steps we need to take to reach the objective, what questions we should answer, and how much time we need. It is a framework based on which you can perform your research in a well-organized and timely manner. In other words, by writing a research proposal, you get a map that shows the direction to the destination (answering the research question). If the proposal is poorly prepared, after spending a lot of energy and money, you may realize that the result of the research has nothing to do with the initial objective, and the study may end up nowhere. Therefore, writing the proposal shows that the researcher is aware of the proper research and can justify the significance of his/her idea.
This is a preview of subscription content, log in via an institution to check access.
Tax calculation will be finalised at checkout
Purchases are for personal use only
Institutional subscriptions
A. Gholipour, E.Y. Lee, S.K. Warfield, The anatomy and art of writing a successful grant application: A practical step-by-step approach. Pediatr. Radiol. 44 (12), 1512–1517 (2014)
Article Google Scholar
L.S. Marshall, Research commentary: Grant writing: Part I first things first …. J. Radiol. Nurs. 31 (4), 154–155 (2012)
E.K. Proctor, B.J. Powell, A.A. Baumann, A.M. Hamilton, R.L. Santens, Writing implementation research grant proposals: Ten key ingredients. Implement. Sci. 7 (1), 96 (2012)
K.C. Chung, M.J. Shauver, Fundamental principles of writing a successful grant proposal. J. Hand Surg. Am. 33 (4), 566–572 (2008)
A.A. Monte, A.M. Libby, Introduction to the specific aims page of a grant proposal. Kline JA, editor. Acad. Emerg. Med. 25 (9), 1042–1047 (2018)
P. Kan, M.R. Levitt, W.J. Mack, R.M. Starke, K.N. Sheth, F.C. Albuquerque, et al., National Institutes of Health grant opportunities for the neurointerventionalist: Preparation and choosing the right mechanism. J. Neurointerv. Surg. 13 (3), 287–289 (2021)
A.M. Goldstein, S. Balaji, A.A. Ghaferi, A. Gosain, M. Maggard-Gibbons, B. Zuckerbraun, et al., An algorithmic approach to an impactful specific aims page. Surgery 169 (4), 816–820 (2021)
S. Engberg, D.Z. Bliss, Writing a grant proposal—Part 1. J. Wound Ostomy Cont. Nurs. 32 (3), 157–162 (2005)
D.Z. Bliss, K. Savik, Writing a grant proposal—Part 2. J. Wound Ostomy Cont. Nurs. 32 (4), 226–229 (2005)
D.Z. Bliss, Writing a grant proposal—Part 6. J. Wound Ostomy Cont. Nurs. 32 (6), 365–367 (2005)
J.C. Liu, M.A. Pynnonen, M. St John, E.L. Rosenthal, M.E. Couch, C.E. Schmalbach, Grant-writing pearls and pitfalls. Otolaryngol. Neck. Surg. 154 (2), 226–232 (2016)
R.J. Santen, E.J. Barrett, H.M. Siragy, L.S. Farhi, L. Fishbein, R.M. Carey, The jewel in the crown: Specific aims section of investigator-initiated grant proposals. J. Endocr. Soc. 1 (9), 1194–1202 (2017)
O.J. Arthurs, Think it through first: Questions to consider in writing a successful grant application. Pediatr. Radiol. 44 (12), 1507–1511 (2014)
M. Monavarian, Basics of scientific and technical writing. MRS Bull. 46 (3), 284–286 (2021)
https://grants.nih.gov
https://grants.nih.gov/grants/oer.htm
https://www.ninr.nih.gov
https://www.niaid.nih.gov
http://www.grantcentral.com
http://www.saem.org/research
http://www.cfda.gov
http://www.ahrq.gov
http://www.nsf/gov
Download references
Authors and affiliations.
School of Dentistry, Marquette University, Milwaukee, WI, USA
Fahimeh Tabatabaei & Lobat Tayebi
You can also search for this author in PubMed Google Scholar
Reprints and permissions
© 2022 Springer Nature Switzerland AG
Tabatabaei, F., Tayebi, L. (2022). Writing a Research Proposal. In: Research Methods in Dentistry. Springer, Cham. https://doi.org/10.1007/978-3-030-98028-3_4
DOI : https://doi.org/10.1007/978-3-030-98028-3_4
Published : 10 April 2022
Publisher Name : Springer, Cham
Print ISBN : 978-3-030-98027-6
Online ISBN : 978-3-030-98028-3
eBook Packages : Engineering Engineering (R0)
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Policies and ethics
A simple explainer with examples + free template.
By: Derek Jansen (MBA) | Reviewed By: Dr Eunice Rautenbach | June 2020 (Updated April 2023)
Whether you’re nearing the end of your degree and your dissertation is on the horizon, or you’re planning to apply for a PhD program, chances are you’ll need to craft a convincing research proposal . If you’re on this page, you’re probably unsure exactly what the research proposal is all about. Well, you’ve come to the right place.
Simply put, a research proposal is a structured, formal document that explains what you plan to research (your research topic), why it’s worth researching (your justification), and how you plan to investigate it (your methodology).
The purpose of the research proposal (its job, so to speak) is to convince your research supervisor, committee or university that your research is suitable (for the requirements of the degree program) and manageable (given the time and resource constraints you will face).
The most important word here is “ convince ” – in other words, your research proposal needs to sell your research idea (to whoever is going to approve it). If it doesn’t convince them (of its suitability and manageability), you’ll need to revise and resubmit . This will cost you valuable time, which will either delay the start of your research or eat into its time allowance (which is bad news).
A good dissertation or thesis proposal needs to cover the “ what “, “ why ” and” how ” of the proposed study. Let’s look at each of these attributes in a little more detail:
Your proposal needs to clearly articulate your research topic . This needs to be specific and unambiguous . Your research topic should make it clear exactly what you plan to research and in what context. Here’s an example of a well-articulated research topic:
An investigation into the factors which impact female Generation Y consumer’s likelihood to promote a specific makeup brand to their peers: a British context
As you can see, this topic is extremely clear. From this one line we can see exactly:
So, make sure that your research proposal provides a detailed explanation of your research topic . If possible, also briefly outline your research aims and objectives , and perhaps even your research questions (although in some cases you’ll only develop these at a later stage). Needless to say, don’t start writing your proposal until you have a clear topic in mind , or you’ll end up waffling and your research proposal will suffer as a result of this.
As we touched on earlier, it’s not good enough to simply propose a research topic – you need to justify why your topic is original . In other words, what makes it unique ? What gap in the current literature does it fill? If it’s simply a rehash of the existing research, it’s probably not going to get approval – it needs to be fresh.
But, originality alone is not enough. Once you’ve ticked that box, you also need to justify why your proposed topic is important . In other words, what value will it add to the world if you achieve your research aims?
As an example, let’s look at the sample research topic we mentioned earlier (factors impacting brand advocacy). In this case, if the research could uncover relevant factors, these findings would be very useful to marketers in the cosmetics industry, and would, therefore, have commercial value . That is a clear justification for the research.
So, when you’re crafting your research proposal, remember that it’s not enough for a topic to simply be unique. It needs to be useful and value-creating – and you need to convey that value in your proposal. If you’re struggling to find a research topic that makes the cut, watch our video covering how to find a research topic .
It’s all good and well to have a great topic that’s original and valuable, but you’re not going to convince anyone to approve it without discussing the practicalities – in other words:
While it’s generally not expected that you’ll have a fully fleshed-out methodology at the proposal stage, you’ll likely still need to provide a high-level overview of your research methodology . Here are some important questions you’ll need to address in your research proposal:
So, be sure to give some thought to the practicalities of your research and have at least a basic methodological plan before you start writing up your proposal. If this all sounds rather intimidating, the video below provides a good introduction to research methodology and the key choices you’ll need to make.
Now that we’ve covered the key points that need to be addressed in a proposal, you may be wondering, “ But how is a research proposal structured? “.
While the exact structure and format required for a research proposal differs from university to university, there are four “essential ingredients” that commonly make up the structure of a research proposal:
In the video below, we unpack each of these four sections, step by step.
In the video below, we provide a detailed walkthrough of two successful research proposals (Master’s and PhD-level), as well as our popular free proposal template.
How long should a research proposal be.
This varies tremendously, depending on the university, the field of study (e.g., social sciences vs natural sciences), and the level of the degree (e.g. undergraduate, Masters or PhD) – so it’s always best to check with your university what their specific requirements are before you start planning your proposal.
As a rough guide, a formal research proposal at Masters-level often ranges between 2000-3000 words, while a PhD-level proposal can be far more detailed, ranging from 5000-8000 words. In some cases, a rough outline of the topic is all that’s needed, while in other cases, universities expect a very detailed proposal that essentially forms the first three chapters of the dissertation or thesis.
The takeaway – be sure to check with your institution before you start writing.
Finding a good research topic is a process that involves multiple steps. We cover the topic ideation process in this video post.
While you typically won’t need a comprehensive literature review at the proposal stage, you still need to demonstrate that you’re familiar with the key literature and are able to synthesise it. We explain the literature review process here.
We explain how to craft a project plan/timeline and budget in Research Proposal Bootcamp .
The expectations and requirements regarding formatting and referencing vary from institution to institution. Therefore, you’ll need to check this information with your university.
We’ve create a video post about some of the most common mistakes students make when writing a proposal – you can access that here . If you’re short on time, here’s a quick summary:
As you write up your research proposal, remember the all-important core purpose: to convince . Your research proposal needs to sell your study in terms of suitability and viability. So, focus on crafting a convincing narrative to ensure a strong proposal.
At the same time, pay close attention to your university’s requirements. While we’ve covered the essentials here, every institution has its own set of expectations and it’s essential that you follow these to maximise your chances of approval.
By the way, we’ve got plenty more resources to help you fast-track your research proposal. Here are some of our most popular resources to get you started:
If you’re looking for 1-on-1 support with your research proposal, be sure to check out our private coaching service , where we hold your hand through the proposal development process (and the entire research journey), step by step.
This post is an extract from our bestselling short course, Research Proposal Bootcamp . If you want to work smart, you don't want to miss this .
I truly enjoyed this video, as it was eye-opening to what I have to do in the preparation of preparing a Research proposal.
I would be interested in getting some coaching.
I real appreciate on your elaboration on how to develop research proposal,the video explains each steps clearly.
Thank you for the video. It really assisted me and my niece. I am a PhD candidate and she is an undergraduate student. It is at times, very difficult to guide a family member but with this video, my job is done.
In view of the above, I welcome more coaching.
Wonderful guidelines, thanks
This is very helpful. Would love to continue even as I prepare for starting my masters next year.
Thanks for the work done, the text was helpful to me
Bundle of thanks to you for the research proposal guide it was really good and useful if it is possible please send me the sample of research proposal
You’re most welcome. We don’t have any research proposals that we can share (the students own the intellectual property), but you might find our research proposal template useful: https://gradcoach.com/research-proposal-template/
Cheruiyot Moses Kipyegon
Thanks alot. It was an eye opener that came timely enough before my imminent proposal defense. Thanks, again
thank you very much your lesson is very interested may God be with you
I am an undergraduate student (First Degree) preparing to write my project,this video and explanation had shed more light to me thanks for your efforts keep it up.
Very useful. I am grateful.
this is a very a good guidance on research proposal, for sure i have learnt something
Wonderful guidelines for writing a research proposal, I am a student of m.phil( education), this guideline is suitable for me. Thanks
You’re welcome 🙂
Thank you, this was so helpful.
A really great and insightful video. It opened my eyes as to how to write a research paper. I would like to receive more guidance for writing my research paper from your esteemed faculty.
Thank you, great insights
Thank you, great insights, thank you so much, feeling edified
Wow thank you, great insights, thanks a lot
Thank you. This is a great insight. I am a student preparing for a PhD program. I am requested to write my Research Proposal as part of what I am required to submit before my unconditional admission. I am grateful having listened to this video which will go a long way in helping me to actually choose a topic of interest and not just any topic as well as to narrow down the topic and be specific about it. I indeed need more of this especially as am trying to choose a topic suitable for a DBA am about embarking on. Thank you once more. The video is indeed helpful.
Have learnt a lot just at the right time. Thank you so much.
thank you very much ,because have learn a lot things concerning research proposal and be blessed u for your time that you providing to help us
Hi. For my MSc medical education research, please evaluate this topic for me: Training Needs Assessment of Faculty in Medical Training Institutions in Kericho and Bomet Counties
I have really learnt a lot based on research proposal and it’s formulation
Thank you. I learn much from the proposal since it is applied
Your effort is much appreciated – you have good articulation.
You have good articulation.
I do applaud your simplified method of explaining the subject matter, which indeed has broaden my understanding of the subject matter. Definitely this would enable me writing a sellable research proposal.
This really helping
Great! I liked your tutoring on how to find a research topic and how to write a research proposal. Precise and concise. Thank you very much. Will certainly share this with my students. Research made simple indeed.
Thank you very much. I an now assist my students effectively.
Thank you very much. I can now assist my students effectively.
I need any research proposal
Thank you for these videos. I will need chapter by chapter assistance in writing my MSc dissertation
Very helpfull
the videos are very good and straight forward
thanks so much for this wonderful presentations, i really enjoyed it to the fullest wish to learn more from you
Thank you very much. I learned a lot from your lecture.
I really enjoy the in-depth knowledge on research proposal you have given. me. You have indeed broaden my understanding and skills. Thank you
interesting session this has equipped me with knowledge as i head for exams in an hour’s time, am sure i get A++
This article was most informative and easy to understand. I now have a good idea of how to write my research proposal.
Thank you very much.
Wow, this literature is very resourceful and interesting to read. I enjoyed it and I intend reading it every now then.
Thank you for the clarity
Thank you. Very helpful.
Thank you very much for this essential piece. I need 1o1 coaching, unfortunately, your service is not available in my country. Anyways, a very important eye-opener. I really enjoyed it. A thumb up to Gradcoach
What is JAM? Please explain.
Thank you so much for these videos. They are extremely helpful! God bless!
very very wonderful…
thank you for the video but i need a written example
Your email address will not be published. Required fields are marked *
Save my name, email, and website in this browser for the next time I comment.
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .
Department of Anaesthesiology, Bangalore Medical College and Research Institute, Bengaluru, Karnataka, India
Writing the proposal of a research work in the present era is a challenging task due to the constantly evolving trends in the qualitative research design and the need to incorporate medical advances into the methodology. The proposal is a detailed plan or ‘blueprint’ for the intended study, and once it is completed, the research project should flow smoothly. Even today, many of the proposals at post-graduate evaluation committees and application proposals for funding are substandard. A search was conducted with keywords such as research proposal, writing proposal and qualitative using search engines, namely, PubMed and Google Scholar, and an attempt has been made to provide broad guidelines for writing a scientifically appropriate research proposal.
A clean, well-thought-out proposal forms the backbone for the research itself and hence becomes the most important step in the process of conduct of research.[ 1 ] The objective of preparing a research proposal would be to obtain approvals from various committees including ethics committee [details under ‘Research methodology II’ section [ Table 1 ] in this issue of IJA) and to request for grants. However, there are very few universally accepted guidelines for preparation of a good quality research proposal. A search was performed with keywords such as research proposal, funding, qualitative and writing proposals using search engines, namely, PubMed, Google Scholar and Scopus.
Five ‘C’s while writing a literature review
A proposal needs to show how your work fits into what is already known about the topic and what new paradigm will it add to the literature, while specifying the question that the research will answer, establishing its significance, and the implications of the answer.[ 2 ] The proposal must be capable of convincing the evaluation committee about the credibility, achievability, practicality and reproducibility (repeatability) of the research design.[ 3 ] Four categories of audience with different expectations may be present in the evaluation committees, namely academic colleagues, policy-makers, practitioners and lay audiences who evaluate the research proposal. Tips for preparation of a good research proposal include; ‘be practical, be persuasive, make broader links, aim for crystal clarity and plan before you write’. A researcher must be balanced, with a realistic understanding of what can be achieved. Being persuasive implies that researcher must be able to convince other researchers, research funding agencies, educational institutions and supervisors that the research is worth getting approval. The aim of the researcher should be clearly stated in simple language that describes the research in a way that non-specialists can comprehend, without use of jargons. The proposal must not only demonstrate that it is based on an intelligent understanding of the existing literature but also show that the writer has thought about the time needed to conduct each stage of the research.[ 4 , 5 ]
The contents or formats of a research proposal vary depending on the requirements of evaluation committee and are generally provided by the evaluation committee or the institution.
In general, a cover page should contain the (i) title of the proposal, (ii) name and affiliation of the researcher (principal investigator) and co-investigators, (iii) institutional affiliation (degree of the investigator and the name of institution where the study will be performed), details of contact such as phone numbers, E-mail id's and lines for signatures of investigators.
The main contents of the proposal may be presented under the following headings: (i) introduction, (ii) review of literature, (iii) aims and objectives, (iv) research design and methods, (v) ethical considerations, (vi) budget, (vii) appendices and (viii) citations.[ 4 ]
It is also sometimes termed as ‘need for study’ or ‘abstract’. Introduction is an initial pitch of an idea; it sets the scene and puts the research in context.[ 6 ] The introduction should be designed to create interest in the reader about the topic and proposal. It should convey to the reader, what you want to do, what necessitates the study and your passion for the topic.[ 7 ] Some questions that can be used to assess the significance of the study are: (i) Who has an interest in the domain of inquiry? (ii) What do we already know about the topic? (iii) What has not been answered adequately in previous research and practice? (iv) How will this research add to knowledge, practice and policy in this area? Some of the evaluation committees, expect the last two questions, elaborated under a separate heading of ‘background and significance’.[ 8 ] Introduction should also contain the hypothesis behind the research design. If hypothesis cannot be constructed, the line of inquiry to be used in the research must be indicated.
It refers to all sources of scientific evidence pertaining to the topic in interest. In the present era of digitalisation and easy accessibility, there is an enormous amount of relevant data available, making it a challenge for the researcher to include all of it in his/her review.[ 9 ] It is crucial to structure this section intelligently so that the reader can grasp the argument related to your study in relation to that of other researchers, while still demonstrating to your readers that your work is original and innovative. It is preferable to summarise each article in a paragraph, highlighting the details pertinent to the topic of interest. The progression of review can move from the more general to the more focused studies, or a historical progression can be used to develop the story, without making it exhaustive.[ 1 ] Literature should include supporting data, disagreements and controversies. Five ‘C's may be kept in mind while writing a literature review[ 10 ] [ Table 1 ].
The research purpose (or goal or aim) gives a broad indication of what the researcher wishes to achieve in the research. The hypothesis to be tested can be the aim of the study. The objectives related to parameters or tools used to achieve the aim are generally categorised as primary and secondary objectives.
The objective here is to convince the reader that the overall research design and methods of analysis will correctly address the research problem and to impress upon the reader that the methodology/sources chosen are appropriate for the specific topic. It should be unmistakably tied to the specific aims of your study.
In this section, the methods and sources used to conduct the research must be discussed, including specific references to sites, databases, key texts or authors that will be indispensable to the project. There should be specific mention about the methodological approaches to be undertaken to gather information, about the techniques to be used to analyse it and about the tests of external validity to which researcher is committed.[ 10 , 11 ]
The components of this section include the following:[ 4 ]
Population refers to all the elements (individuals, objects or substances) that meet certain criteria for inclusion in a given universe,[ 12 ] and sample refers to subset of population which meets the inclusion criteria for enrolment into the study. The inclusion and exclusion criteria should be clearly defined. The details pertaining to sample size are discussed in the article “Sample size calculation: Basic priniciples” published in this issue of IJA.
The researcher is expected to give a detailed account of the methodology adopted for collection of data, which include the time frame required for the research. The methodology should be tested for its validity and ensure that, in pursuit of achieving the results, the participant's life is not jeopardised. The author should anticipate and acknowledge any potential barrier and pitfall in carrying out the research design and explain plans to address them, thereby avoiding lacunae due to incomplete data collection. If the researcher is planning to acquire data through interviews or questionnaires, copy of the questions used for the same should be attached as an annexure with the proposal.
This addresses the strength of the research with respect to its neutrality, consistency and applicability. Rigor must be reflected throughout the proposal.
It refers to the robustness of a research method against bias. The author should convey the measures taken to avoid bias, viz. blinding and randomisation, in an elaborate way, thus ensuring that the result obtained from the adopted method is purely as chance and not influenced by other confounding variables.
Consistency considers whether the findings will be consistent if the inquiry was replicated with the same participants and in a similar context. This can be achieved by adopting standard and universally accepted methods and scales.
Applicability refers to the degree to which the findings can be applied to different contexts and groups.[ 13 ]
This section deals with the reduction and reconstruction of data and its analysis including sample size calculation. The researcher is expected to explain the steps adopted for coding and sorting the data obtained. Various tests to be used to analyse the data for its robustness, significance should be clearly stated. Author should also mention the names of statistician and suitable software which will be used in due course of data analysis and their contribution to data analysis and sample calculation.[ 9 ]
Medical research introduces special moral and ethical problems that are not usually encountered by other researchers during data collection, and hence, the researcher should take special care in ensuring that ethical standards are met. Ethical considerations refer to the protection of the participants' rights (right to self-determination, right to privacy, right to autonomy and confidentiality, right to fair treatment and right to protection from discomfort and harm), obtaining informed consent and the institutional review process (ethical approval). The researcher needs to provide adequate information on each of these aspects.
Informed consent needs to be obtained from the participants (details discussed in further chapters), as well as the research site and the relevant authorities.
When the researcher prepares a research budget, he/she should predict and cost all aspects of the research and then add an additional allowance for unpredictable disasters, delays and rising costs. All items in the budget should be justified.
Appendices are documents that support the proposal and application. The appendices will be specific for each proposal but documents that are usually required include informed consent form, supporting documents, questionnaires, measurement tools and patient information of the study in layman's language.
As with any scholarly research paper, you must cite the sources you used in composing your proposal. Although the words ‘references and bibliography’ are different, they are used interchangeably. It refers to all references cited in the research proposal.
Successful, qualitative research proposals should communicate the researcher's knowledge of the field and method and convey the emergent nature of the qualitative design. The proposal should follow a discernible logic from the introduction to presentation of the appendices.
Conflicts of interest.
There are no conflicts of interest.
Writing an evaluation plan.
An evaluation plan is an integral part of a grant proposal that provides information to improve a project during development and implementation.
For small projects, the Office of the Vice President for Research can help you develop a simple evaluation plan. If you are writing a proposal for larger center grant, using a professional external evaluator is recommended. We can provide recommendations of external evaluators; please contact [email protected] ; for BioMed faculty visit the BioMed Evaluation Services webpage .
Do all grant proposals require an evaluation plan?
Not all grant proposals require an evaluation plan; however, many program announcements and funding opportunities stipulate and evaluation strategy with specific milestones are important elements that should be considered. If an evaluation plan is required, it will generally be listed in the program announcement. Most often, larger, more involved grant proposals will require an evaluation plan, while a smaller, single-investigator proposals will not. If you are unsure whether your proposal requires an evaluation plan, please contact us.
It is worth noting there is a difference between evaluation and research although there are several commonalities. Most simply:
There are two types of evaluation typically requested by funders--formative and summative—and which you use is largely dictated by the purpose of the evaluation. Do you want to prove that you achieved the outcomes as intended (summative) or are you doing evaluation to monitor if you are doing what you said you would in your grant application (formative)? Or both? We can help you prepare and review both types of evaluations outlined below.
Formative or Process Evaluation does the following:
Summative or Outcomes Evaluation does the following:
What does the evaluation process entail?
The evaluation process can be broken down into a series of steps, from preparation to implementation and interpretation.
Home Market Research
Content Index
Qualitative methods.
Evaluation research, also known as program evaluation, refers to research purpose instead of a specific method. Evaluation research is the systematic assessment of the worth or merit of time, money, effort and resources spent in order to achieve a goal.
Evaluation research is closely related to but slightly different from more conventional social research . It uses many of the same methods used in traditional social research, but because it takes place within an organizational context, it requires team skills, interpersonal skills, management skills, political smartness, and other research skills that social research does not need much. Evaluation research also requires one to keep in mind the interests of the stakeholders.
Evaluation research is a type of applied research, and so it is intended to have some real-world effect. Many methods like surveys and experiments can be used to do evaluation research. The process of evaluation research consisting of data analysis and reporting is a rigorous, systematic process that involves collecting data about organizations, processes, projects, services, and/or resources. Evaluation research enhances knowledge and decision-making, and leads to practical applications.
LEARN ABOUT: Action Research
The common goal of most evaluations is to extract meaningful information from the audience and provide valuable insights to evaluators such as sponsors, donors, client-groups, administrators, staff, and other relevant constituencies. Most often, feedback is perceived value as useful if it helps in decision-making. However, evaluation research does not always create an impact that can be applied anywhere else, sometimes they fail to influence short-term decisions. It is also equally true that initially, it might seem to not have any influence, but can have a delayed impact when the situation is more favorable. In spite of this, there is a general agreement that the major goal of evaluation research should be to improve decision-making through the systematic utilization of measurable feedback.
Evaluation Research lets you understand what works and what doesn’t, where we were, where we are and where we are headed towards. You can find out the areas of improvement and identify strengths. So, it will help you to figure out what do you need to focus more on and if there are any threats to your business. You can also find out if there are currently hidden sectors in the market that are yet untapped.
It is essential to gauge your past performance and understand what went wrong in order to deliver better services to your customers. Unless it is a two-way communication, there is no way to improve on what you have to offer. Evaluation research gives an opportunity to your employees and customers to express how they feel and if there’s anything they would like to change. It also lets you modify or adopt a practice such that it increases the chances of success.
After evaluating the efforts, you can see how well you are meeting objectives and targets. Evaluations let you measure if the intended benefits are really reaching the targeted audience and if yes, then how effectively.
Evaluations help you to analyze the demand pattern and predict if you will need more funds, upgrade skills and improve the efficiency of operations. It lets you find the gaps in the production to delivery chain and possible ways to fill them.
All market research methods involve collecting and analyzing the data, making decisions about the validity of the information and deriving relevant inferences from it. Evaluation research comprises of planning, conducting and analyzing the results which include the use of data collection techniques and applying statistical methods.
Some of the evaluation methods which are quite popular are input measurement, output or performance measurement, impact or outcomes assessment, quality assessment, process evaluation, benchmarking, standards, cost analysis, organizational effectiveness, program evaluation methods, and LIS-centered methods. There are also a few types of evaluations that do not always result in a meaningful assessment such as descriptive studies, formative evaluations, and implementation analysis. Evaluation research is more about information-processing and feedback functions of evaluation.
These methods can be broadly classified as quantitative and qualitative methods.
The outcome of the quantitative research methods is an answer to the questions below and is used to measure anything tangible.
The best way to collect quantitative data is through surveys , questionnaires , and polls . You can also create pre-tests and post-tests, review existing documents and databases or gather clinical data.
Surveys are used to gather opinions, feedback or ideas of your employees or customers and consist of various question types . They can be conducted by a person face-to-face or by telephone, by mail, or online. Online surveys do not require the intervention of any human and are far more efficient and practical. You can see the survey results on dashboard of research tools and dig deeper using filter criteria based on various factors such as age, gender, location, etc. You can also keep survey logic such as branching, quotas, chain survey, looping, etc in the survey questions and reduce the time to both create and respond to the donor survey . You can also generate a number of reports that involve statistical formulae and present data that can be readily absorbed in the meetings. To learn more about how research tool works and whether it is suitable for you, sign up for a free account now.
Create a free account!
Quantitative data measure the depth and breadth of an initiative, for instance, the number of people who participated in the non-profit event, the number of people who enrolled for a new course at the university. Quantitative data collected before and after a program can show its results and impact.
The accuracy of quantitative data to be used for evaluation research depends on how well the sample represents the population, the ease of analysis, and their consistency. Quantitative methods can fail if the questions are not framed correctly and not distributed to the right audience. Also, quantitative data do not provide an understanding of the context and may not be apt for complex issues.
Learn more: Quantitative Market Research: The Complete Guide
Qualitative research methods are used where quantitative methods cannot solve the research problem , i.e. they are used to measure intangible values. They answer questions such as
LEARN ABOUT: Qualitative Interview
Qualitative data is collected through observation, interviews, case studies, and focus groups. The steps for creating a qualitative study involve examining, comparing and contrasting, and understanding patterns. Analysts conclude after identification of themes, clustering similar data, and finally reducing to points that make sense.
Observations may help explain behaviors as well as the social context that is generally not discovered by quantitative methods. Observations of behavior and body language can be done by watching a participant, recording audio or video. Structured interviews can be conducted with people alone or in a group under controlled conditions, or they may be asked open-ended qualitative research questions . Qualitative research methods are also used to understand a person’s perceptions and motivations.
LEARN ABOUT: Social Communication Questionnaire
The strength of this method is that group discussion can provide ideas and stimulate memories with topics cascading as discussion occurs. The accuracy of qualitative data depends on how well contextual data explains complex issues and complements quantitative data. It helps get the answer of “why” and “how”, after getting an answer to “what”. The limitations of qualitative data for evaluation research are that they are subjective, time-consuming, costly and difficult to analyze and interpret.
Learn more: Qualitative Market Research: The Complete Guide
Survey software can be used for both the evaluation research methods. You can use above sample questions for evaluation research and send a survey in minutes using research software. Using a tool for research simplifies the process right from creating a survey, importing contacts, distributing the survey and generating reports that aid in research.
Evaluation research questions lay the foundation of a successful evaluation. They define the topics that will be evaluated. Keeping evaluation questions ready not only saves time and money, but also makes it easier to decide what data to collect, how to analyze it, and how to report it.
Evaluation research questions must be developed and agreed on in the planning stage, however, ready-made research templates can also be used.
Jun 18, 2024
Jun 17, 2024
Other categories.
Not all studies should be treated equally. Below are a few key factors to consider when evaluating a study’s conclusions.
Comparing proposals “apples-to-apples” is crucial to establishing which one will best meet your needs. Consider these ideas to help you focus on the details that contribute to a successful survey.
The proposal process begins well before you ask any research firm for quote. The process really begins with the discussions you and your team have about objectives. What are your goals? What are the decisions you want to make when the project is done and you have data in hand?
Once you have a solid vision of the survey, then it’s time to start talking with potential partners Throughout your conversations, take note: Do the various firms ask you specific questions about your objectives, the group of people you’d like to survey, and your ultimate goals? Do they, indeed, ask about decisions that you wish to make? Details regarding your specific need should always be front and center during the conversations.
When reviewing the sampling plan, make sure the proposal mentions sample size, response rate estimates, number of responses, and maximum sampling error. If you’re unsure of the impact these figures have on the quality of your results, ask the researcher. They should be able to explain them in terms you can understand.
The quantity and types of information sought from respondents will impact cost. Quantity encompasses the number of questions and number of variables to process. Type refers to how the questions will be processed, the data entry involved and whether all or just some data will be cleaned.
No evaluation is complete until you know the approximate number and types of questions planned for the survey. The number of open-ended questions should be included as well because open-ended questions that capture verbatim responses can impact the response rate and possibly the price of your survey, especially if done by mail.
In addition, make sure the proposal clearly indicates who will develop the questionnaire content. Also, determine if it includes enough collaboration time to be sufficiently customized to meet your particular needs.
For online surveys paying attention to the data collection series and who is responsible for sending survey invitations. Multiple emails to sample members can encourage response. As well, the invitation process should be sensitive to data privacy issues such as those indicated by GDPR and others. Proposals for mailed surveys should clearly outline the data collection series and each component of the survey kit.
Any proposal you receive should highlight the steps the research company will take to make sure that the data is accurate and representative. Depending on the type of survey, checking logic, consistency, and outliers can take a significant amount of time. You must have some process noted to identify inconsistent answers for surveys that collect a significant amount of numerical data (salary survey, market studies, budget planning). Finally, some percentage of mailed surveys need to be verified for data entry accuracy.
A straightforward analysis of survey data can meet many objectives. In other cases, a multivariate statistical analysis will provide deeper insights to achieve your objectives— making results easier to use. If your objectives include learning about separate segments of your circulation, crosstabulations should be specified.
A variety of reporting options exist for a survey. These include but are not limited to data tables, a summary of the results, in-depth analysis, and graphed presentations. As a result, you need to understand exactly what you’ll receive following your survey and in what format.
Make sure the proposal covers all the bases: what you need to do and provide, what the firm will do when they will do it and how much it will cost. There should be no surprises in what you need to supply. No “you need how much letterhead and envelopes?” a week before your survey is scheduled to mail. Review the price carefully and understand what it includes and doesn’t include. As with many things in life, you usually get what you pay for.
Related posts:, notes on the pre- and post-survey.
Notes on the Pre- and Post-Survey The Pre-Post survey is used to look at how things may change over time. It may be how brand awareness levels change after a new ad campaign is introduced or how opinions of a political candidate move after a speech. The catalyst to potential change, sometimes called the event […]
The Importance of Questionnaire Design Planning a survey requires many steps and decisions along the way: “How many people do I need to survey? How am I going to distribute the survey?” And, while people often figure out what questions they want to ask, many overlook the importance of expert, unbiased questionnaire design. When you […]
Will Color Printing Give Your Survey a Boost? Occasionally we are asked if color printing (versus black and white) is better to use as part of a survey mailing? Will this treatment generate more attention and ultimately a better response? Our Opinion: If you’re looking to use color to boost your survey’s response rate, it […]
Your email address will not be published. Required fields are marked *
This site uses Akismet to reduce spam. Learn how your comment data is processed .
Improving the science process skills of science, technology, engineering students through personality-based approach, 4 references, the research student's guide to success, doing your research project: a guide for first-time researchers, research methodology: a step-by-step guide for beginners, research methodology: methods and techniques, related papers.
Showing 1 through 3 of 0 Related Papers
Eduinput- Online tutoring platform for Math, chemistry, Biology Physics
An online learning platform for Mcat, JEE, NEET and UPSC students
Table of Contents
A research proposal is a document that outlines the plan and rationale for conducting a research study. It serves as a blueprint for the entire research process and helps researchers communicate their objectives, methods, and expected outcomes effectively.
The key components of a research proposal include:
Also learn about Action Proposal
Research proposals can vary depending on the field of study and the intended audience. Different types of research proposals can help you determine which format is most appropriate for your specific needs.
Whether responding to a solicitation, submitting an unsolicited proposal, or seeking continuation or renewal funding, each proposal type requires careful consideration and alignment with the sponsor’s objectives and guidelines.
Here are some common types of research proposals:
Solicited proposals are submitted in response to a specific call or request issued by a sponsor. These calls, often referred to as Request for Proposals (RFP) or Request for Quotations (RFQ), outline the sponsor’s specific requirements, objectives, and evaluation criteria.
Solicited proposals must adhere to the provided guidelines and may include technical specifications and terms and conditions set by the sponsor. Broad Agency Announcements (BAAs) are similar but are not considered formal solicitations.
Unsolicited proposals are submitted to a sponsor without a specific request or solicitation. In these cases, the investigator believes that the sponsor has an interest in the subject matter. Unsolicited proposals require the researcher to present a compelling case for the significance and relevance of their research, convincing the sponsor of the value and potential impact of the proposed study.
Preproposals are typically requested by sponsors who want to streamline the application process and minimize the effort required by applicants. Preproposals are in the form of a letter of intent or a brief abstract that outlines the main objectives and approach of the research.
After reviewing the preproposal, the sponsor informs the investigator if a full proposal is warranted. This process allows both the investigator and the sponsor to determine if it is worthwhile to proceed with a complete proposal submission.
Continuation or non-competing proposals are submitted for multi-year projects that have already received funding from the sponsor for an initial period, typically one year. These proposals confirm the original proposal’s scope, objectives, and funding requirements for the subsequent period.
The sponsor’s decision to continue funding is contingent upon satisfactory work progress and the availability of funds.
Renewal or competing proposals are submitted when an existing project is nearing its end, and the investigator requests continued support for the research. From the sponsor’s perspective, these proposals are treated similarly to unsolicited proposals, requiring a thorough presentation of the project’s achievements, impact, and future plans.
Renewal proposals must demonstrate the ongoing relevance and value of the research, highlighting the need for further funding to continue the project’s objectives.
Grant proposals are submitted to funding agencies, such as government bodies, foundations, or organizations, to secure financial support for research projects.
These proposals typically require a detailed description of the research project, including the objectives, methodology, expected outcomes, budget, and timeline. Grant proposals often follow specific guidelines provided by the funding agency.
Dissertation proposals are submitted by doctoral students as part of their research journey. These proposals outline the research topic, objectives, theoretical framework, methodology, and anticipated contributions to the field.
Dissertation proposals also typically include a literature review to establish the context and significance of the proposed research.
Project proposals are common in academic and professional settings where research projects are undertaken. These proposals outline the objectives, scope, methodology, timeline, and expected outcomes of the project.
Project proposals often include details about the project team, resources required, and the potential impact of the project on stakeholders.
Similar to dissertation proposals, thesis proposals are submitted by students pursuing a master’s degree. These proposals present the research topic, objectives, methodology, and expected contributions to the field.
Thesis proposals also include a literature review that highlights the existing knowledge and research gaps in the chosen area of study.
Research funding proposals are typically submitted by researchers or research teams within academic institutions or research organizations. These proposals aim to secure funding for ongoing or new research projects.
Research funding proposals often include a detailed description of the research objectives, methodology, expected outcomes, budget, and timeline. They may also require a justification for the need for funding and a demonstration of the potential impact of the research.
Feasibility study proposals are used to assess the practicality and viability of a research project before its full implementation. These proposals outline the research objectives, methodology, timeline, and expected outcomes, with a particular focus on evaluating the feasibility of conducting the research.
Feasibility study proposals often involve preliminary data collection or analysis to inform the decision-making process.
Program evaluation proposals are designed to assess the effectiveness, efficiency, and impact of a specific program, intervention, or policy. These proposals typically outline the evaluation objectives, methodology, data collection methods, analysis techniques, and expected outcomes.
Program evaluation proposals often require collaboration with relevant stakeholders and may involve both qualitative and quantitative research methods.
Following steps are involved in Developing a Research Proposal:
Selecting a suitable research topic is important for the success of your research proposal. Consider the following tips when choosing your research topic:
Crafting a well-structured and compelling research proposal is essential for several reasons:
Research proposals serve as essential tools for planning and initiating research projects across various fields. They play a crucial role in academic, scientific, and professional settings. Here are some key applications of research proposals:
Your email address will not be published. Required fields are marked *
Save my name, email, and website in this browser for the next time I comment.
Get updates about new courses
Join our scholarship program
Click Here to join
9th Class 10th Class 11 Class 12 Class
Join the groups below to get updates.
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.
Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR 1999;48 (No. RR-11)
This Public Health Reports article highlights the path CDC has taken to foster the use of evaluation. Access this valuable resource to learn more about using evaluation to inform program improvements.
Evaluation: A systematic method for collecting, analyzing, and using data to examine the effectiveness and efficiency of programs and, as importantly, to contribute to continuous program improvement.
Program: Any set of related activities undertaken to achieve an intended outcome; any organized public health action. At CDC, program is defined broadly to include policies; interventions; environmental, systems, and media initiatives; and other efforts. It also encompasses preparedness efforts as well as research, capacity, and infrastructure efforts.
At CDC, effective program evaluation is a systematic way to improve and account for public health actions.
We have to have a healthy obsession with impact. To always be asking ourselves what is the real impact of our work on improving health? Dr. Frieden, January 21, 2014
Research seeks to prove, evaluation seeks to improve. Michael Quinn Patton, Founder and Director of Utilization-Focused Evaluation
E-mail: [email protected]
To receive email updates about this page, enter your email address:
IMAGES
VIDEO
COMMENTS
The research journey commences with the selection of a research topic and the preparation of a proposal on the selected topic. Experience has shown that students tend to encounter difficulties in writing research proposals for their supervisors because they do not fully comprehend what constitutes a research proposal.
1.1. Theoretical Framework. The first step has been the clear definition of what is an evaluation strategy or methodology.The term evaluation is defined by the Cambridge Dictionary as the process of judging something's quality, importance, or value, or a report that includes this information [] or in a similar way by the Oxford Dictionary as the making of a judgment about the amount, number ...
Definition: Evaluating Research refers to the process of assessing the quality, credibility, and relevance of a research study or project. This involves examining the methods, data, and results of the research in order to determine its validity, reliability, and usefulness. Evaluating research can be done by both experts and non-experts in the ...
on research assessment, especially the assessment of researchers (as opposed to research proposals). This is important for this discussion. When we say we judge the excellence of the proposal or researcher, we do not expect the application to satisfy each element of a broad portfolio of demands.
To assess research proposals, funders rely on the services of peer experts to review the thousands or perhaps millions of research proposals seeking funding each year. While often associated with scholarly publishing, peer review also includes the ex ante assessment of research grant and fellowship applications ( Abdoul et al. 2012 ).
INTRODUCTION. Creativity and critical thinking are of particular importance in scientific research. Basically, research is original investigation undertaken to gain knowledge and understand concepts in major subject areas of specialization, and includes the generation of ideas and information leading to new or substantially improved scientific insights with relevance to the needs of society.
The peer review of research proposals (grants) aims to judge the merit of projects and researchers and enable the best to be contemplated. The director of an institution in the United Kingdom shared on Twitter his struggle in evaluating the numerous proposals received and started a discussion forum from which ideas and suggestions emerged.
Research proposal examples. Writing a research proposal can be quite challenging, but a good starting point could be to look at some examples. We've included a few for you below. Example research proposal #1: "A Conceptual Framework for Scheduling Constraint Management".
A research proposal is a roadmap that brings the researcher closer to the objectives, takes the research topic from a purely subjective mind, and manifests an objective plan. It shows us what steps we need to take to reach the objective, what questions we should answer, and how much time we need. It is a framework based on which you can perform ...
The purpose of the research proposal (its job, so to speak) is to convince your research supervisor, committee or university that your research is suitable (for the requirements of the degree program) and manageable (given the time and resource constraints you will face). The most important word here is "convince" - in other words, your ...
Dimensions of the quality of research practice. Evaluation of the quality of research practice is a truly important issue in most scientific domains and at many levels (European Science Foundation, 2012). Increasingly, we are also seeing these assessment efforts across disciplinary and national boundaries.
A proposal needs to show how your work fits into what is already known about the topic and what new paradigm will it add to the literature, while specifying the question that the research will answer, establishing its significance, and the implications of the answer. [ 2] The proposal must be capable of convincing the evaluation committee about ...
Writing an Evaluation Plan. An evaluation plan is an integral part of a grant proposal that provides information to improve a project during development and implementation. For small projects, the Office of the Vice President for Research can help you develop a simple evaluation plan. If you are writing a proposal for larger center grant, using ...
The process of evaluation research consisting of data analysis and reporting is a rigorous, systematic process that involves collecting data about organizations, processes, projects, services, and/or resources. Evaluation research enhances knowledge and decision-making, and leads to practical applications. LEARN ABOUT: Action Research.
Source Definition; Suchman (1968, pp. 2-3) [Evaluation applies] the methods of science to action programs in order to obtain objective and valid measures of what such programs are accomplishing.…Evaluation research asks about the kinds of change desired, the means by which this change is to be brought about, and the signs by which such changes can be recognized.
The research proposal serves a triple goal, namely explaining why there is a for your need research, detailing why your research is feasible and perspectives your research. As such writing a research proposal is a valuable exercise even if you do not pursue a scientific career. You will have to
its importance to the field of special. education, and its relationship to prior research and literature. 2. o Your evaluation of' the personnel and their ability to assume responsibility in this particular area of inquicyo Secondly, a:re ·th~ facilities adequate for this task? 3. o . The . adequacy . of the research design and evaluation o:f ...
Peer-review, the process by which a study is sent to other researchers in a particular field for their notes and thoughts, is essential in evaluating a study's findings. Since most consumers and members of the media are not well-trained enough to evaluate a study's design and researcher's findings, studies that pass muster with other ...
Comparing proposals "apples-to-apples" is crucial to establishing which one will best meet your needs. Consider these ideas to help you focus on the details that contribute to a successful survey. Make sure the proposal responds to your objectives. The proposal process begins well before you ask any research firm for quote.
A high quality proposal not only promises success for the project, but also impresses the Thesis Jury about the student's potential as a researcher. A research proposal is intended to convince others that the student has a worthwhile research project and that s/he has the competence and the work-plan to complete it. sydney.edu.au. Save to ...
Evaluation provides the words, the stories, and the statistics to garner more support and broaden and deepen your impact. 6) Produces results you can trust. At the end of the day, you want to know that what you are doing is working. Furthermore, you want to be sure you are making a difference and you need to know how you are changing lives for ...
II-The importance of research proposal A research proposal is a document of usually ten to fifteen pages that informs others of a proposed piece of research. ... It shows the student's ability to critically evaluate relevant literature information. • 4. It indicates the student's ability to integrate and ...
These proposals outline the research objectives, methodology, timeline, and expected outcomes, with a particular focus on evaluating the feasibility of conducting the research. Feasibility study proposals often involve preliminary data collection or analysis to inform the decision-making process.
Evaluation: Purpose is to determine effectiveness of a specific program or model and understand why a program may or may not be working. Goal is to improve programs. Research: Purpose is theory testing and to produce generalizable knowledge. Goal is to contribute to knowledge base. Monitoring: Purpose is to track implementation progress through periodic data collection.