Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

  • We're Hiring!
  • Help Center

paper cover thumbnail

"Measuring the Effectiveness of Performance Management Systems: A Comprehensive Review and Framework"

Profile image of Dr. Venkateswararao Podile

2019, IJFANS International Journal of Food and Nutritional Sciences

This research endeavors to comprehensively evaluate and enhance the effectiveness of performance management systems (PMS) within organizational contexts. Through an extensive review of existing PMS across diverse industries, this study analyzes employee perceptions, organizational outcomes, and the role of leadership in shaping PMS efficacy. Key performance metrics are identified, encompassing both quantitative and qualitative indicators, to gauge system success. Additionally, the study explores the integration of innovative technologies in PMS and assesses their impact. The findings culminate in the development of a comprehensive framework, providing organizations with practical guidelines for measuring, improving, and optimizing their performance management systems.

Related Papers

International Journal of Academic Management Science Research (IJAMSR)

David E Amanawa

Abstract: This paper discusses the theoretical underpinnings, analysis, design, and implementation of performance tools and systems. Performance management tools are software applications that track individual, team, or organizational productivity. In contrast, a performance management system is a standardized process that provides feedback to management on how an organization’s employees perform. Theories of performance management serve as the foundation for performance management. They are crucial to organizational behavior and human resource management because they assist employers in leading and motivating their workforces. A critical evaluation of what performance tools and system entails revealed that it is a way of helping employees and managers achieve their goals and improve their work quality. According to the results, it is advised to consider the following ideas: Managers should encourage staff members to speak up and provide regular feedback to their managers. They should use goal-setting tools to help them define targets and key results and match their own goals with those of the organization. You may create a culture of trust and ongoing improvement by using this strategy. Finally, managers should recognize and praise employees socially for any contribution that they have made to the organization.

research paper on performance management system

Amitabh Upadhya

This paper through literature review attempts to explore different types of Performance Management Systems prevalent in organizations across the UAE and to find out its influence on employees for better performance. The paper proposes an extensive organized assessment of PMS design on the basis of established theories; presenting both concepts and frameworks that feature on the ground. It also suggests for more studies on a complete PMS framework and screening of PMS related to development of both, competence and innovation resulting in long-term employee retention. By analyzing various PMS designs it is concluded that PMS has to be designed effectively in such a way that is contingent on both internal and external variables. Both the financial performance measures and non-financial performance measures are assessed together, the link between PMS and Strategic Organizational objectives should be ratified through various methods of PM mechanisms.

Seetha Pachchhapur

This paper focuses on the perceptions and experiences of employees in two different large organisations related to Performance Management Systems (PMS) in their respective organisations. Drawing on individual employee interview data, the research paper presents a comparative assessment of PMS and reports the interpretation of employee data under various themes pertaining to design and implementation considerations. The results suggest that PMS perceptions vary due to design anomalies and an incongruity in the implementation of PMS practices which has a bearing on the effectiveness and strength of the system. In considering PMS theory and practices, organisation intentions and actions, the paper seeks to engage in debates emerging on the organisation’s dependence on PMS process for achieving goals and also developing employees.

Journal of Applied Psychology

Junhyok Yim

eli suherli

Performance management systems are designed to improve employee performance. However, the introduction of the Balance Scorecard redesigned the objectives of the performance management system. This study examines the moderating effect of using Balance Scorecard on the impact of performance management systems on employee performance. This research was conducted at manufacturing companies listed on the stock exchange in Indonesia. The research tool is a survey that includes 192 top managers from 63 registered manufacturing companies. The findings show that the use of Balance Scorecard moderates the impact of performance management systems on employee performance. This implies that the use of Balance Scorecard strategically complements and enhances the relationship between employee performance and performance management systems. This study shows that the Balance Scorecard must be used as a multi-dimensional method for measuring performance and as a strategic management system to improve...

C. Jordaan , Cecile Schultz

Knowledge of employees’ perceptions of the performance management system will assist management to implement such a system more effectively. The purpose of the study was to determine the perceptions of employees regarding the effectiveness of the performance management system at a government department in South Africa. A quantitative research approach was employed and a questionnaire was distributed to 1 200 employees at the department under study. The response rate was 44%. A factor analysis resulted in three main factors, namely personal development, personal performance, and manager support. Descriptive statistics and the ANOVA were performed to determine the perceptions of employees on these three factors. Negativity towards personal development and manager support, and positivity towards personal performance were detected.

International Journal of Productivity and Performance Management

Andre de Waal

Developing Country Studies

dolores hervie

Performance management system (PMS) is the heart of any “people management” process in an organization. Organizations exist to perform. If employees do not perform, organizations would not survive. If employees perform at their peak level, organizations would be competitive and productivity would increase. Performance management supports an organisation in the achievement of strategic goals and objectives. It assists managers in setting clear performance expectation goals for all employees that are aligned with the strategic goals. It provides a structure for employees to develop the skills they need to achieve the organisation’s strategic goals. The key features of PMS involve setting objectives, measurable performance criteria expected of the individual employee must be made known to him/her from the beginning of the year. There is regular day-to-day meeting as well as periodic formal feedback meetings with subordinates. An unbiased annual assessment of performance based on speci...

Increasingly, organizations are finding themselves in operational environments that are more dynamic and resource scarce; as a result, many organizations are focusing on becoming more agile and competitive through organizational improvement initiatives. One major facilitator of these improvement initiatives is performance measurement (PM) which has experienced an increase in both academic focus and practitioner application. Although scholarly research has been conducted in the design, implementation, use, and continuous improvement of PM systems, the majority of applications in the literature report less than optimal results or, in some cases, failure of the PM system. In the past decade, research into the implementation phase of PM systems has identified many factors for both success and failure in an attempt to increase the likelihood that the full benefits from these systems will be realized. This paper characterizes the factors identified in the literature by means of systematic literature review, remarks on the maturity of this line of research, and identifies areas for future research.

Studies in Business and Economics

Mihaela HERCIU

The utility of performance measurement and management system can be said to have been proven, but the problem faced by both the theoreticians and practitioners is to set the right performance indicators. Developed models are tools that managers can use to measure and manage performance, but they need to be tailored to the context. Also, the trend towards using non-financial performance indicators makes it very difficult for managers to design a performance measurement and management system because it involves the integration of qualitative and non-quantitative variables and a profound understanding of the internal and external environment of the company.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Aditya Gautam

Asia Pacific Journal of Human Resources

tricia vilkinas

Management Accounting Research

Aldonio Ferreira , David Otley

Alisa Hutchinson

Harini Perera

Dr. Sukanta Mishra

Operations Management Research

Roman A . Lewandowski

Ramyadevi Arumugham

Dickson Mdhlalose

International Journal of Business …

Dr. Bishnu Sharma

Neelam Gupta

iaeme iaeme

Industrial Engineering Letters

Chandan Kumar Sahoo

KnE Social Sciences

Fatimah Mahmud

Isuri Kumari

Adriana Buhovac , Maja Groff

Dr. Sukanta Mishra, Doctorate in Commerce & Management

International Business & Economics Research Journal (IBER)

Prakash Singh

Aditya Gautam , Sameeksha Jain

IJSRD - International Journal for Scientific Research and Development

Measuring Business Excellence

Kashi Balachandran , Flavio Tonelli

RELATED TOPICS

  •   We're Hiring!
  •   Help Center
  • Find new research papers in:
  • Health Sciences
  • Earth Sciences
  • Cognitive Science
  • Mathematics
  • Computer Science
  • Academia ©2024

Performance management systems, innovative work behavior and the role of transformational leadership: an experimental approach

Journal of Organizational Effectiveness: People and Performance

ISSN : 2051-6614

Article publication date: 14 March 2023

Issue publication date: 19 March 2024

Despite increasing attention to employee development, past research has mostly studied performance management systems (PMSs) in relation to task-related behaviors compared to proactive behaviors. Accordingly, this study addresses the relation between PMSs and innovative work behavior (IWB).

Design/methodology/approach

Building on signaling theory and human resource management (HRM) system strength research, the authors designed a factorial survey experiment ( n  = 444) to examine whether PMSs stimulate IWB under different configurations of distinctiveness, consistency and consensus, as well as in the presence of transformational leadership.

Results show that only strong PMSs foster IWB (high distinctiveness, high consistency and high consensus [HHH]). Additional analyses reveal that the individual meta-features of PMS consistency and consensus can also stimulate innovation. Transformational leadership reinforced the relationship between PMS consensus and IWB relationship, but not the relationships of the other meta-features.

Practical implications

The study’s findings suggest that organizations wishing to unlock employees' innovative potential should design PMSs that are visible, comprehensible and relevant. To further reap the innovative gains of employees, organizations could also invest in the coherent and fair application of planning, feedback and evaluation throughout the organization and ensure organizational stakeholders agree on the approach to PMSs.

Originality/value

The study’s findings show that PMS can also inspire proactivity in employees, in the form of IWB and suggest that particular leadership behaviors can complement certain PMS meta-features, and simultaneously also compete with PMS strength, suggesting the whole (i.e. PMS strength) is more than the sum of the parts (i.e. PMS meta-features).

  • Innovative work behavior
  • Performance management system

Transformational leadership

  • Signaling theory

Bauwens, R. , Audenaert, M. and Decramer, A. (2024), "Performance management systems, innovative work behavior and the role of transformational leadership: an experimental approach", Journal of Organizational Effectiveness: People and Performance , Vol. 11 No. 1, pp. 178-195. https://doi.org/10.1108/JOEPP-03-2022-0066

Emerald Publishing Limited

Copyright © 2023, Robin Bauwens, Mieke Audenaert and Adelien Decramer

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Introduction

Challenges like digitalization, financial crises and pandemics render innovation essential for organizational survival. Since employees are an important source of innovation, the question becomes how to stimulate innovative work behavior (IWB), being employees' proactive behavior in creating and applying novel ideas at work ( Prieto and Pérez-Santana, 2014 ; Mustafa et al. , 2021 ). Research on human resource management (HRM) and IWB has highlighted a variety of HRM arrangements that benefit innovation ( Bos-Nehles et al. , 2017a ), like high-performance work systems (HPWSs; Do et al., 2019 ) or reward systems ( Hussain et al. , 2019 ). Yet, while some studies have focused on innovation in relation to individual practices of performance management systems (PMSs), like performance appraisal (e.g.  Botelho, 2020 ; Curzi et al. , 2019 ; Singh et al. , 2021 ) or how to innovate PMSs themselves (e.g. Anh Vu et al. , 2022 ), few studies have addressed the relationship between PMSs and IWB ( Audenaert et al. , 2019 ). This is problematic, because PMSs constitute a fundamental cornerstone of HRM ( Albrecht et al. , 2015 ; DeNisi and Murphy, 2017 ).

PMSs refer to a go-together of planning, feedback and evaluation activities that “give[s] employees the means, motivation, and opportunity to improve firm-level performance” ( Schleicher et al. , 2018 , p. 2211). Despite their importance, PMSs have so far mainly been studied in relation to task-related behaviors ( Berdicchia et al. , 2022 ). For long time, the idea that PMSs can also stimulate proactive behaviors, which by definition are self-initiated behaviors, has been dismissed over criticisms that PMSs are often reduced to administrative chores, disconnected from day-to-day activities and with little motivational value ( Mertens et al. , 2021 ; Murphy, 2020 ). While recent insights suggest that some PMS activities do have proactive potential, few studies have made the case for when and how PMSs foster proactive behaviors, like IWB ( Berdicchia et al. , 2022 ; Van Veldhoven et al. , 2017 ). Addressing this gap is important given that PMSs are gradually evolving from a results-oriented focus towards a development-oriented focus concerned with a more diverse range of positive employee outcomes ( Aguinis et al. , 2012 ; Bizri et al. , 2021 ; Kubiak, 2022 ; Van Veldhoven et al. , 2017 ).

This paper examines the relationship between PMSs and IWB. In doing, we make two main contributions by arguing that fundamental to understanding this relationship are (1) employees' perceptions ( Van Waeyenberg et al. , 2022 ) and (2) leaders' involvement in the implementation of PMSs ( Lee et al. , 2020 ). First, HRM-innovation research has traditionally devoted little attention to employee perceptions, which are nonetheless fundamental to truly grasp how employees experience and act upon HRM ( Bos-Nehles and Veenendaal, 2019 ). The present paper incorporates employees' perceptions by combining signaling theory ( Connelly et al. , 2011 ) with the HRM system strength concept ( Bowen and Ostroff, 2004 ). Signaling theory is increasingly valued as a theoretical lens to comprehend how people react to HRM ( Guest et al. , 2021 ) and PMSs in extension ( Bauwens et al. , 2019 ; Biron et al. , 2011 ). Signaling theory states that HRM instruments, like PMSs, reflect organizational signals about values, expectations and rewards that employees use as a basis for their behavior. The concept of HRM strength states that such signals are easier to interpret when HRM instruments are distinct (i.e. visible, understandable, relevant and backed by legitimate authority), consistent (i.e. instrumental, valid and coherent messages) and consensual (i.e. fair and agreed upon) ( Presbitero et al. , 2022 ). The extent to which PMS displays these three “meta-features” of HRM system strength (i.e. distinctiveness, consistency, consensus) is referred to in PMS research as “PMS strength” (cf. Van Thielen et al., 2022 ; Van Waeyenberg et al. , 2022 ). We propose that PMS strength influences the extent to which employees will align their behavior with the signals sent out by PMSs and that this also applies for proactive behaviors, like IWB. Importantly, while past research asserts that all three meta-features are required for employees to align their behavior, our study also examines whether different configurations of PMS strength (meta-features) could be equally effective in stimulating IWB. In doing, we contribute to recent configurational developments in the HRM system strength literature (cf.  Aksoy and Bayazit, 2014 ; Bos-Nehles et al. , 2021 ; Sanders et al. , 2021 ) and to a better understanding of how these meta-features are related to one another.

Second, little is known about the boundary conditions of HRM system strength ( Presbitero et al. , 2022 ). Despite a growing body of research on HRM system strength that has committed itself to PMSs, the same is true for PMS strength (e.g. Van Thielen et al. , 2022 ; Van Waeyenberg et al. , 2022 ). Indeed, well-designed PMSs alone will not always lead to the desired behaviors. There is an increased understanding that the outcomes of PMSs depend on the leadership of line managers, responsible for their implementation ( Lee et al. , 2020 ). Accordingly, we examine whether leadership influences the extent to which PMS strength motivates employees to engage in IWB. We focus on transformational leadership, since past research demonstrates that its combination of vision, support and intellectual stimulation not only interacts with PMSs ( Campbell et al. , 2016 ), but also inspires employees to go beyond requirements ( Audenaert et al. , 2019 ).

In making these contributions, we adopt an experimental approach in which we manipulate PMS strength configurations through different experimental scenarios. This methodological innovation responds to recent calls in the field for more credible research designs ( Sanders et al. , 2021 ). We add to the emerging body of experimental knowledge on HRM perceptions (e.g. Batistič and Poell, 2022 ; Flinchbaugh et al., 2020 ; Meier‐Barthold et al. , 2023 ; Sanders and Yang, 2016 ) and to that of PMS perceptions specifically ( Van Thielen et al. , 2022 ), which could aid scholars to establish causal links and combat endogeneity problems in PMS and HRM system strength research.

A signaling approach to performance management system strength and innovative work behavior

PMSs are a go-together of planning, feedback and evaluation activities that help employees to attain performance expectations ( Kubiak, 2022 ; Schleicher et al. , 2018 ). Because those performance expectations are assumed to benefit firm performance, PMS research has mostly focused on task-related behaviors ( Berdicchia et al. , 2022 ). However, organizational PMSs are gradually abandoning their narrow result-oriented focus with an emphasis on compliance in favor of a development-oriented focus. The latter entails focusing on a broader behavioral repertoire that also includes more proactive behaviors, like IWB ( Aguinis et al. , 2012 ; Van Veldhoven et al. , 2017 ; Kubiak, 2022 ). Accordingly, this study examines the relationship between PMSs and IWB, which we define as employees' proactive behavior in creating and applying novel ideas at work. We argue that the relationship between PMSs and IWB can be explained through a combination of signaling theory ( Connelly et al. , 2011 ) and the concept of HRM system strength ( Bowen and Ostroff, 2004 ).

Signaling theory is concerned with communication in organizations ( Connelly et al. , 2011 ). It considers HRM arrangements as ways in which organizations convey values, expectations and rewards to employees, who are assumed to align their behavior accordingly. In doing, this theory refines some of the mechanisms behind attributional approaches like HRM system strength ( Bowen and Ostroff, 2004 ) that “address the quality and strength of the [HRM] signal” ( Guest et al., 2021 , p. 798). Following the logic of signaling theory, PMSs are used by organizations to alter employee attitudes and behaviors. Through planning, feedback and evaluation, PMSs communicate expectations that employees interpret as signals to which they need to commit their behavior ( Biron et al. , 2011 ; Bednall et al. , 2022 ).

Employees' interpretation of these signals does not only motivate task-related behaviors and performances, but can also inspire proactive ones, like IWB. According to Parker et al. (2010) , employees engage in proactive behavior when they believe they can successfully engage in such behavior (“can do motivation”), see the value of such behavior (“reason to motivation”) and/or experience positive affective states (“energized to” motivation). A recent study by Berdicchia et al. (2022) shows that PMSs trigger “can do” and “reason to” motivations. From a signaling theory perspective, this implies that PMSs could signal expectations that help employees to see the value and success of IWB, ultimately motivating them to engage in such behavior. However, the extent to which employees will act upon these signals depends on the strength of these signals ( Guest et al. , 2021 ).

Employee perceptions of PMS strength are positively and significantly related to IWB.

Employee perceptions of PMS strength configurations high in distinctiveness and consistency are positively and significantly related to IWB.

The moderating role of transformational leadership

Line managers also send signals that can “amplify” those sent out by PMSs and increase the chance that employees will act upon signaled expectations ( Bauwens et al. , 2019 ). Line managers can do this in several ways. For example, through verbal clarification, role modeling and positive reinforcement of desired behaviors or by engaging in a dialogue with employees over their comprehension of signaled expectations ( Nishii and Paluch, 2018 ). A particular leadership approach that captures such line manager behaviors, but also stimulates employees to go beyond job requirements is transformational leadership ( Kou et al., 2022 ).

Transformational leadership moderates the relationship between employee perceptions of PMS strength and IWB, in such a way that the relationship is stronger under high transformational leadership.

Figure 1 provides a graphical summary of the hypothesized relationships.

Materials and methods

Data collection took place through Prolific Academic ( www.prolific.co ). We recruited 444 participants for a small fee (£9/hr). All participants were employed and reported to a supervisor. Participants worked in a variety of industries like business services (19.60%), retail (13.50%) healthcare (10.10%), Government (13.90%) and Production (11.70%). The mean age of the respondents was 28.91 years (SD = 8.56). The majority were male (52.30%) and possessed at least a bachelor's degree (46.40%).

Participants completed an online survey (Qualtrics) with validated scales and a descriptive experimental scenario. In the first part of the survey, participants completed the control variables and rated their supervisor on transformational leadership. In the second part of the survey, participants were presented with an experimental scenario in which we manipulated the three PMS strength meta-features (distinctiveness, consistency, consensus) as either high or low, resulting in eight different versions of the scenario (2 × 2 × 2) that were randomly assigned to respondents. After the scenario, we conducted a manipulation check. Finally, we asked employees to what extent they would engage in IWB if they and their supervisor found themselves in the presented scenario.

We used the scale by Carless et al. (2000) , rated on a five-point scale (1 = strongly disagree; 5 = strongly agree). A sample item is “My supervisor communicates a clear and positive vision of the future.” Cronbach's alpha was 0.91.

PMS strength manipulations

Imagine you find yourself in the following situation. At the beginning of the month, you have a meeting with your supervisor. During this meeting, your supervisor clarifies (1) what is expected of you this month, (2) when you will receive feedback and (3) based on which criteria your success will be evaluated.
Based your observations, your impression of this meeting is that your supervisor’s explanation is [H: clear and understandable/L: unclear and confusing]. [L: Nonetheless, agreements are made]. You think your supervisor’s approach to goal setting, feedback and evaluation will [H: enable/L: hinder] you to deliver upon your expectations and grow in your role. You also believe your supervisor has [H: sufficient/L: insufficient] expertise and experience to oversee this process.
Furthermore, you learn that other supervisors in your organization use [H: the same approach/L: very different approaches] to managing employees.
That month you and your supervisor meet weekly. Both the interim feedback and final evaluation you receive [H: align/L: conflict] with the earlier communication and agreements made with your supervisor. At the end of the month, you perform exceptionally well. You attribute this achievement mainly to [H: your supervisor’s approach to goal setting, feedback, and evaluation/ L: yourself]. Your overall feeling is that you [H: get/ L: do not get] the recognition you deserve.

Manipulation check

After reading the scenario, participants completed a manipulation check by rating the scenario on a nine-item PMS strength scale adapted Van Waeyenberg and Decramer (2018) , which past research has used to assess PMS strength and its meta-features (1 = strongly disagree; 5 = strongly agree). A sample item is “In the scenario, the supervisor's approach to goal-setting, feedback and appraisal was accompanied by a clear consistency between words and actions”.

After completing the manipulation check, we asked participants to what extent they would feel motivated to engage in IWB in the subsequent weeks if they found themselves in the displayed shown scenario with their supervisor. Participants had previously rated their supervisor on transformational leadership, and we instructed them to keep this rating in mind when picturing themselves in the scenario and answering the IWB-related questions. To assess participants' IWB we used a scale by Bos-Nehles and Veenendaal (2019) which combines the dimensions of opportunity exploration (“paying attention to non-routine issues in your work, department, organization, or the marketplace”), idea generation (“generating original solutions to problems”), championing (“attempting to convince people to support an innovative idea”) and application (“contributing to the implementation of new ideas”). Answers were rated on a five-point scale (1 = to a very small extent; 5 = to a very large extent). Despite these multiple dimensions, confirmatory factor analysis (CFA) determined a one-dimensional IWB scale a good fit to the data ( χ 2  = 85.07, df = 40, CFI = 0.99, RMSEA = 0.05), while a four-dimensional IWB scale presented no significant improvement (Δ χ 2  = 6.80, Δdf = 2., p  = 0.07). Cronbach's alpha for the overall scale was 0.93.

Control variables

To avoid extraneous variables affecting experimental outcomes, participants were randomly assigned to one of the eight experimental groups. Nevertheless, experimental groups can still differ in composition and affect outcome variables, which necessitates statistical control. Therefore, we controlled for gender (0 = female, 1 = male), age (in years) and education (primary education, secondary education, bachelor, master, PhD). Past research shows that IWB is higher among men ( Sanders et al. , 2018 ) and highly educated employees ( Sanders and Yang, 2016 ), but decreases with age ( Curzi et al. , 2019 ). We also accounted for sector (0 = private, 1 = public and non-profit) as Bos-Nehles et al. (2017b) make note of significant discrepancies in the HRM-innovation linkage across different sectors.

Manipulation checks

To test the effectiveness of our experimental manipulation, we compared the PMS strength scores from the manipulation check with PMS strength in the different scenarios through a series of one-way ANOVA's. The results show that respondents perceived significantly more PMS distinctiveness in high-distinctiveness scenarios ( F (1, 442) = 284.36; p  < 0.001), PMS consistency in high-consistency scenarios ( F (1, 442) = 43.10; p  < 0.001) and PMS consensus in high-consensus scenarios ( F (1, 442) = 72.79; p  < 0.001). Therefore, we conclude our manipulation worked as expected.

Descriptive statistics and correlations

Table 1 shows the descriptive statistics and correlations. IWB showed a positive relationship with transformational leadership ( r  = 0.25, p  < 0.01), the HHH scenario ( r  = 0.21, p  < 0.01) and the female gender ( r  = 0.14, p  < 0.01). IWB also showed a negative association with the HLL scenario ( r  = −0.19, p  < 0.01). All correlations remained below the threshold of |0.80| and variance inflation factors (VIF) stayed below 7.00 (range 1.20–2.40), indicating no multicollinearity concerns.

Hypothesis tests: PMS strength configurations

Table 2 displays the regression results. We first calculated a model which isolates the effects of the experimental scenarios (model 1), and subsequently added the main effects and control variables (model 2), and finally the interactions (model 3). In the models, we used the LLL-scenario as reference category since this scenario corresponds to the absence of strong PMS activities. In line with H1a , employees reported more IWB in the HHH scenario ( b  = 0.40, p  < 0.001). Contrary to H1b , we did not find indications for alternative PMS strength configurations, like those high in distinctiveness and consistency, stimulating IWB. Contrary to H2 , transformational leadership did not reinforce IWB in the HHH scenario ( b  = −0.09, p  = 0.45), but only moderated the relationship between PMS strength and IWB in the LLH scenario ( b  = 0.35, p  < 0.05).

Additional analysis: PMS strength meta-features

While the hypothesis tests did not reveal alternative PMS strength configurations capable of stimulating IWB, authors like Van Waeyenberg et al. (2022) suggest that different PMS “meta-features” might still exhibit a differential impact on PMS outcomes, like IWB. To that end, we conducted an additional analysis in which we recoded the eight PMS strength scenarios into three dummy variables reflecting PMS distinctiveness (1 = high), consistency (1 = high) and consensus (1 = high). Table 3 displays these results. In partial support of H1b , we found that IWB benefits from scenarios high in PMS consistency ( b  = 0.18, p  < 0.01) and PMS consensus ( b  = 0.24, p  < 0.001), but not from scenarios high in PMS distinctiveness ( b  = −0.10, p  > 0.05). In partial support of H2 , transformational leadership positively moderated the relationship between PMS consensus and IWB ( b  = 0.17, p  < 0.05), but not the relationships of the other two PMS strength meta-features. Figure 2 displays the interaction plot for the relationship between PMS consensus and IWB for high (+1SD) and low (−1SD) transformational leadership. It shows that the relationship between PMS consensus and IWB is stronger when transformational leadership is high.

The present study set out to look at employees' perceptions of PMS strength, IWB and the moderating role of transformational leadership. To investigate these relationships, we employed experimental scenarios embedded in a survey. Three main findings emerged from our research.

First, our study revealed that IWB benefits from PMSs where all meta-features are high (i.e. HHH configurations). This is in line with signaling theory ( Connelly et al. , 2011 ) and the traditional view of HRM system strength ( Bowen and Ostroff, 2004 ). When PMS strength is high, PMSs send out stronger signals that increase the chance that (1) employees will pick up these signals and (2) commit their behavior accordingly ( Guest et al. , 2021 ). Second, we found that PMS meta-features can have differential relationships with PMS outcomes. While we did not find indications of differential PMS strength configurations as advanced by authors like Aksoy and Bayazit (2014) or Bos-Nehles et al. (2021) , participants reported more IWB when confronted with scenarios high in PMS consistency and PMS consensus, but remained indifferent to PMS distinctiveness. This is in line with past research on PMS consistency (e.g.  Audenaert et al. , 2019 ; Bauwens et al. , 2019 ; Van Thielen et al. , 2018 ), but runs counter to studies that have found stronger effects for distinctiveness compared to other PMS strength meta-features (e.g. Aksoy and Bayazit, 2014 ; Van Waeyenberg and Decramer, 2018 ). A potential explanation is that proactive behaviors, like IWB, benefit from HRM systems that resemble strong commitment configurations ( Batistič et al. , 2022 ). PMSs high in consistency and consensus might compare to strong commitment configurations because they stimulate employees' internalization of organizational expectations through consistent HRM messages, organizational agreement and fairness principles. However, such systems might lack the legitimacy and relevance of PMSs high in distinctiveness to enforce those specific expectations. Consequently, employees will internalize the organizational expectations and engage in IWB to make proactive contributions to organizational goals.

Third, in line with studies that have endorsed leadership as a contingency of PMSs and their outcomes (e.g, Audenaert et al. , 2019 ; Lee et al. , 2020 ), we found that transformational leadership reinforced the relationship between PMS consistency and IWB. However, the same could not be observed for other PMS configurations or meta-features. On the one hand, this suggests that transformational leaders' charismatic and championing behaviors serve as catalysts for employees to unite behind a common PMS approach. On the other hand, it could also imply that clear, legitimate and fully coherent PMSs might direct employees' attention away from transformational leaders' visionary characteristics and, as a result, could leave such leaders little leeway to reinforce the innovative potential of PMSs.

Theoretical implications

This research makes two main theoretical contributions to the literature on HRM and IWB. The first contribution concerns how employees' perceptions of PMSs relate to IWB. By (a) combining signaling theory ( Connelly et al. , 2011 ) with Parker et al. ’s (2010) proactive motivation model and by (b) demonstrating that strong PMS inspire IWB, this study provides a theoretical mechanism through which PMSs can inspire proactive behaviors without setting specific proactive goals (cf. Ligon et al ., 2012 ) and move beyond being compliance-oriented systems. That is, strong HRM systems, like PMSs, send signals about organizational values, expectations and rewards. In turn, those signals aid employees in believing they can successfully engage in proactive behavior, in seeing the value of such behavior and/or in feeling energized by the prospect of engaging in such behavior. As such, this study advances the emerging literature on PMSs and proactivity (e.g. Berdicchia et al ., 2022 ; Van Veldhoven et al ., 2017 ). Furthermore, by looking at both PMS meta-features (PMS distinctiveness, consistency, consensus) and configurations of such features (HHH, HHL, HLH, HLL, LLL, LHL, LLH, LHH), this study highlights the merits of a strong PMS, as well as the individual meta-features of PMS consistency and consensus. Together, these results show that strong PMSs is more than the sum of its parts. As such, this study extends recent debates in HRM system strength literature (cf. Aksoy and Bayazit, 2014 ; Bos-Nehles et al ., 2021 ; Sanders et al. , 2021 ) to PMS research.

A second contribution deals with transformational leadership as a boundary condition of PMS strength. This study has taken a signaling theory approach to line managers' transformational leadership because it has considered leadership as an “amplifier” of the signals sent out by PMSs (cf. Bauwens et al. , 2019 ; Lee et al. , 2020 ). By demonstrating that a specific leadership style (i.e. transformational leadership) only reinforces specific meta-features (i.e. PMS consensus) and that the presence of strong PMSs can direct employees' attention away from particular leader behaviors, this study suggests that specific leadership styles might act as competing mechanisms for PMSs, while other leadership styles might act as complementary mechanisms to PMSs to pursue proactive behaviors like IWB (cf. Audenaert et al. , 2019 ; Campbell et al. , 2016 ). Overall, such findings suggest that PMSs and leaders work together in more complex ways. To untangle this complex interplay, further studies that investigate potential interactions of leadership and PMS characteristics are necessary. In this sense, scholars like Leroy et al. (2018) draw attention to different patterns of “leader-HRM fit”. That is, line manager leadership could not only moderate, but can also predict (i.e. dynamic fit) and mediate the relationship between PMS strength and its outcomes (i.e. enactment). For example, past research shows that PMS strength is predicted by differences in line managers' ability, motivation and opportunity ( Van Waeyenberg and Decramer, 2018 ). Therefore, it is up to future studies to investigate different types of “leader-PMS fit” to further unravel the leadership styles and behaviors that unlock the proactive, innovative potential of PMSs.

Limitations

This study contains limitations. First, the experimental vignettes did not incorporate transformational leadership and IWB but measured them indirectly via employee reports. This might explain why the results for transformational leadership were rather modest, as some perspective-taking from the side of the participant was required, which uses more cognitive resources. In a similar manner, future experiments could also assess IWB through real-life innovative tasks incorporated into the experimental design (e.g. brainstorm task). Second, our operationalization in specific vignettes focused on individual perceptions of the PMS process (i.e. how PMSs take place). Employees might react differently when PMS content (i.e. specific goals or practices) is also included in the experimental manipulation. This could be, for example, in the form of specific innovation goals ( Ligon et al. , 2012 ), through developmental goal-setting that considers employees' unique talents and strengths ( van Woerkom and Kroon, 2020 ) or through a well-developed feedback culture ( Mertens et al. , 2021 ). Employees might react differently to PMSs in a team context, for example by discussing shared goals and possibilities to attain them. As more and more organizations draw on teams and team-based working, it becomes essential for PMSs to not only focus on individual development, but also on team development. This fosters collaboration and prevents internal competition. Therefore, future experimental PMS studies could design scenarios where both PMS process and content are combined, while also considering the multilevel nature of (team) PMS perceptions ( Van Thielen et al., 2018 ).

Implications for policy and practice

Our study conveys three important messages to organizations wishing to unlock their employees' innovation. First, by linking PMSs to IWB, our study shows that the benefits of PMSs are not limited to compliance and task performance. Instead, PMSs can also stimulate proactive behaviors, like IWB and therefore also serve more developmental purposes. Second, organization should continue to invest in strong PMSs. Strong PMSs can be achieved by (1) providing planning, feedback and evaluation activities that are visible, comprehensible and relatable to employees and their job (this stimulates IWB by convincing employees of the value and need for such behavior); (2) investing in the coherent application of planning, feedback and evaluation throughout the organization (this ensures the underlying message is reinforced, increasing the chance that employees will pick it up and align their behavior); (3) ensuring fairness and agreement from organizational stakeholders on PMS approaches (this provides employees with psychological safety and a stronger belief in the success of their innovative attempts). Despite assertions that there could be multiple ways in which PMSs can achieve their outcomes, our study found no support for such alternatives. Instead, we highlighted the merits of strong PMSs over their alternatives. This implies that the three points above are not an either-or story, but a full approach to PMS implementation. Finally, organizations should select and develop visionary and charismatic leaders with idealized influence, inspirational motivation, individual consideration and intellectual stimulation. Through role modeling, appealing to a collective identity and increased personal consideration for employees, such leaders enhance the likelihood that employees will respond in an innovative way to the signaled expectations of strong PMSs. Our additional analyses show that this is especially the case when the implementation of PMSs is coherent and enjoys agreement, as clear PMSs might restrain such leaders.

PMSs are reflections of what organizations consider important, which employees use as input to their own behavior. Through a survey with experimental scenarios, this study demonstrated that PMSs represent a source for self-initiated, proactive behaviors, like IWB. While employees also use leader behavior as input, our findings for transformational leadership were mixed, which could suggest that strong PMSs could also provide less leeway for line managers. Overall, this study highlights that PMSs are not merely compliance-oriented HRM arrangements, but that they can also serve as motivators for innovation.

Conceptual model

Interaction plot for the relation between PMS consistency and IWB for high (+1SD) and low (−1SD) transformational leadership

MeanSD1234567891011121314
1Age28.918.56
2Gender (1 = female)0.480.500.01
3Sector (1 = public and nonprofit)0.230.420.070.12*
4Education2.960.790.20**0.020.11*
5PMS strength: LLL0.140.35−0.020.03−0.10*−0.01
6PMS strength: LHL0.130.340.050.02−0.020.09−0.02**
7PMS strength: LHH0.120.32−0.07−0.030.08−0.03−0.15**−0.14**
8PMS strength: LLH0.140.34−0.01−0.07−0.010.08−0.16**−0.16**−0.15**
9PMS strength: HHL0.120.320.010.05−0.01−0.05−0.15**−0.14**−0.13**−0.15**
10PMS strength: HLH0.110.310.04−0.010.030.01−0.14**−0.13**−0.13**−0.14**−0.13**
11PMS strength: HLL0.120.33−0.020.020.04−0.08−0.15**−0.15**−0.14**−0.15**−0.14**−0.13**
12PMS strength: HHH0.120.330.01−0.01−0.01−0.02−0.15**−0.15**−0.14**−0.15**−0.14**−0.13**−0.14**
13Transformational leadership3.770.78−0.16**0.050.070.01−0.010.04−0.050.060.010.02−0.090.02
14IWB3.570.75−0.0180.14**0.050.010.01−0.070.03−0.040.010.06−0.19**0.21**0.25**
*  < 0.05; **  < 0.01; ***  < 0.00;  = 444

Authors' own work

Model 1Model 2Model 3
SE SE SE
Age 0.010.01−0.010.00
Employee gender (1 = female) 0.18**0.070.20**0.07
Sector (1 = public and nonprofit) 0.060.080.050.09
Education −0.010.04−0.010.05
Transformational leadership 0.25***0.040.27***0.05
LHL−0.060.14−0.080.13−0.080.13
LLH−0.040.15−0.050.13−0.070.14
HHL−0.020.14−0.010.140.010.13
HLH−0.130.140.100.140.110.13
LHH0.080.130.100.140.110.12
HLL−0.32*0.15−0.28**0.13−0.240.15
HHH0.41***0.110.40***0.130.40***0.11
LHL × Transformational leadership 0.0120.20
LLH × Transformational leadership 0.35*0.17
HHL × Transformational leadership −0.200.14
HLH × Transformational leadership 0.160.14
LHH × Transformational leadership 0.0170.18
HLL × Transformational leadership 0.180.17
HHH × Transformational leadership −0.090.13
F5.19***6.55***4.88***
0.070.160.19
Adjusted 0.050.130.07
*  < 0.05; **  < 0.01***  < 0.00. For the interactions and main effects, the LLL scenario (i.e. absence of PMS strength) serves as reference category;  = 444

Authors' own work

IWB
SE
Age0.010.00
Employee gender (1 = female)0.21**0.07
Sector (1 = public and nonprofit)0.010.09
Education−0.010.05
Transformational leadership0.28***0.05
PMS distinctiveness0.060.13
PMS consistency0.18**0.14
PMS consensus0.24***0.13
Transformational leadership × PMS Distinctiveness−0.100.20
LLH × Transformational leadership × PMS Consistency−0.160.17
HHL × Transformational leadership × PMS Consensus0.17*0.14
F6.41***
0.14
Adjusted 0.12

Note(s): * p  < 0.05; ** p  < 0.01*** p  < 0.00; n  = 444

Source(s): Authors' own work

Transformational leadership ( Carless et al., 2000 )

… communicates a clear and positive vision of the future.

… treats staff as individuals, supports and encourages their development.

… gives encouragement and recognition to staff.

… fosters trust, involvement, and cooperation among team members.

… encourages thinking about problems in new ways and questions assumptions.

… is clear about his/her values and practices which he/she preaches.

… instils pride and respect in others and inspires me by being highly competent.

Manipulation check ( Van Waeyenberg and Decramer, 2018 )

To what extent do the following statements apply to the previously shown scenario?

In the scenario, the supervisor's approach to goal-setting, feedback and appraisal …

Distinctiveness

… was easy to understand.

… was appreciated.

… was experienced as relevant.

Consistency

… contributed to better functioning.

… succeeded in reinforcing the desired behavior and realizes the goals for which it was intended and designed.

… was accompanied by a clear consistency between the words and actions of my supervisor.

… was based on mutual agreement between supervisors in the organization about how to deal with employees.

… was accompanied by impartial decisions by my supervisor.

… was considered fair.

Innovative work behavior ( Bos-Nehles and Veenendaal, 2019 )

If you found yourself in the previously shown scenario with your supervisor, to what extent would you feel motivated to engage in the following behaviors in the subsequent weeks?

If I found myself with my supervisor in this scenario, I would feel motivated in the subsequent weeks to …

Opportunity exploration

… pay attention to non-routine issues in my work, department, organization, or the market place.

… look for opportunities to improve an existing process, technology, product, service or work relationship.

… recognize opportunities to make a positive difference in my work, department, organization, or with customers.

Idea generation

… search out new working methods, techniques, or instruments.

… generate original solutions to problems.

… find new approaches to execute tasks.

Championing

… attempt to convince people to support an innovative idea.

… make important organizational members enthusiastic for innovative ideas.

Application

… put effort into the development of new things.

… contribute to the implementation of new ideas.

… systematically introduce innovative ideas into work practices.

Aguinis , H. , Gottfredson , R. and Joo , H. ( 2012 ), “ Delivering effective performance feedback: the strengths-based approach ”, Business Horizons , Vol.  55 No.  2 , pp.  105 - 111 , doi: 10.1016/j.bushor.2011.10.004 .

Aksoy , E. and Bayazit , M. ( 2014 ), “ The relationships between MBO system strength and goal‐climate quality and strength ”, Human Resource Management , Vol.  53 No.  4 , pp.  505 - 525 , doi: 10.1002/hrm.21603 .

Albrecht , S. , Bakker , A. , Gruman , J. , Macey , W. and Saks , A. ( 2015 ), “ Employee engagement, human resource management practices and competitive advantage: an integrated approach ”, Journal of Organizational Effectiveness , Vol.  2 No.  1 , pp.  7 - 35 , doi: 10.1108/JOEPP-08-2014-0042 .

Anh Vu , T. , Plimmer , G. , Berman , E. and Ha , P. ( 2022 ), “ Performance management in the Vietnam public sector: the role of institution, traditional culture and leadership ”, International Journal of Public Administration , Vol.  45 No.  1 , pp.  49 - 63 , doi: 10.1080/01900692.2021.1903499 .

Audenaert , M. , Decramer , A. , George , B. , Verschuere , B. and Van Waeyenberg , T. ( 2019 ), “ When employee performance management affects individual innovation in public organizations: the role of consistency and LMX ”, International Journal of Human Resource Management , Vol.  30 No.  5 , pp.  815 - 834 , doi: 10.1080/09585192.2016.1239220 .

Bass , B. and Riggio , R. ( 2006 ), Transformational Leadership , Lawrence Erlbaum , Mahwah, NJ .

Batistič , S. and Poell , R. ( 2022 ), “ Do HR systems and relational climates affect knowledge hiding? An experiment and two-source multi-level study ”, Journal of Business Research , Vol.  147 , pp.  82 - 96 , doi: 10.1016/j.jbusres.2022.04.016 .

Batistič , S. , Kenda , R. , Premru , M. and Černe , M. ( 2022 ), “ HR systems and leadership attachment affecting idea generation and implementation: an experiment and two-source multi-level study ”, European Management Journal , Vol.  40 No.  4 , pp.  532 - 545 , doi: 10.1016/j.emj.2021.09.005 .

Bauwens , R. , Audenaert , M. and Decramer , A. ( 2019 ), “ Fostering societal impact and job satisfaction: the role of performance management and leader-member exchange ”, Public Management Review , Vol.  21 No.  10 , pp.  1486 - 1515 , doi: 10.1080/14719037.2018.1561928 .

Bednall , T. , Sanders , K. and Yang , H. ( 2022 ), “ A meta‐analysis on employee perceptions of human resource strength: examining the mediating versus moderating hypotheses ”, Human Resource Management , Vol.  61 No.  1 , pp.  5 - 20 , doi: 10.1002/hrm.22068 .

Berdicchia , D. , Bracci , E. and Masino , G. ( 2022 ), “ Performance management systems promote job crafting: the role of employees' motivation ”, Personnel Review , Vol.  51 No.  3 , pp.  861 - 875 , doi: 10.1108/PR-05-2020-0361 .

Biron , M. , Farndale , E. and Paauwe , J. ( 2011 ), “ Performance management effectiveness: lessons from world-leading firms ”, International Journal of Human Resource Management , Vol.  22 No.  6 , pp.  1294 - 1311 , doi: 10.1080/09585192.2011.559100 .

Bizri , R. , Wahbi , M. and Al Jardali , H. ( 2021 ), “ The impact of CSR best practices on job performance: the mediating roles of affective commitment and work engagement ”, Journal of Organizational Effectiveness , Vol.  8 No.  1 , pp.  129 - 148 , doi: 10.1108/JOEPP-01-2020-0015 .

Bos-Nehles , A. and Veenendaal , A. ( 2019 ), “ Perceptions of HR practices and innovative work behavior: the moderating effect of an innovative climate ”, International Journal of Human Resource Management , Vol.  30 No.  18 , pp.  2661 - 2683 , doi: 10.1080/09585192.2017.1380680 .

Bos-Nehles , A. , Bondarouk , T. and Nijenhuis , K. ( 2017a ), “ Innovative work behaviour in knowledge-intensive public sector organizations: the case of supervisors in The Netherlands fire services ”, International Journal Of Human Resource Management , Vol.  28 No.  2 , pp.  379 - 398 , doi: 10.1080/09585192.2016.1244894 .

Bos-Nehles , A. , Renkema , M. and Janssen , M. ( 2017b ), “ HRM and innovative work behaviour: a systematic literature review ”, Personnel Review , Vol.  46 No.  7 , pp.  1228 - 1253 , doi: 10.1108/PR-09-2016-0257 .

Bos-Nehles , A. , Conway , E. and Fox , G. ( 2021 ), “ Optimising human resource system strength in nurturing affective commitment: do all meta-features matter? ”, Human Resource Management Journal , Vol.  31 No.  2 , pp.  493 - 513 , doi: 10.1111/1748-8583.12320 .

Botelho , C. ( 2020 ), “ The influence of organizational culture and HRM on building innovative capability ”, International Journal of Productivity and Performance Management , Vol.  69 No.  7 , pp.  1373 - 1393 , doi: 10.1108/IJPPM-05-2019-0228 .

Bowen , D. and Ostroff , C. ( 2004 ), “ Understanding HRM–firm performance linkages: the role of the “strength” of the HRM system ”, Academy of Management Review , Vol.  29 No.  2 , pp.  203 - 221 , doi: 10.5465/AMR.2004.12736076 .

Campbell , J. , Lee , H. and Im , T. ( 2016 ), “ At the Expense of others: altruistic helping behaviour, performance management and transformational leadership ”, Public Management Review , Vol.  18 No.  6 , pp.  795 - 818 , doi: 10.1080/14719037.2015.1045018 .

Carless , S. , Wearing , A. and Mann , L. ( 2000 ), “ A short measure of transformational leadership ”, Journal of Business and Psychology , Vol.  14 No.  3 , pp.  389 - 405 , doi: 10.1023/A:1022991115523 .

Connelly , B. , Certo , S. , Ireland , R. and Reutzel , C. ( 2011 ), “ Signaling theory: a review and assessment ”, Journal of Management , Vol.  37 No.  1 , pp.  39 - 67 , doi: 10.1177/0149206310388419 .

Curzi , Y. , Fabbri , T. , Scapolan , A. and Boscolo , S. ( 2019 ), “ Performance appraisal and innovative behavior in the digital era ”, Frontiers in Psychology , Vol.  10 , p. 1659 , doi: 10.3389/fpsyg.2019.01659 .

DeNisi , A. and Murphy , K. ( 2017 ), “ Performance appraisal and performance management: 100 years of progress? ”, Journal of Applied Psychology , Vol.  102 No.  3 , pp.  421 - 433 , doi: 10.1037/apl0000085 .

Do , H. , Budhwar , P. and Patel , C. ( 2019 ), “ High-performance work system practices in Vietnam: a study of managers' perceptions ”, Journal of Organizational Effectiveness , Vol.  6 No.  3 , pp.  145 - 160 , doi: 10.1108/JOEPP-07-2018-0048 .

Flinchbaugh , C. , Ishqaidef , G. and Chadwick , C. ( 2020 ), “ A shared human resources change initiative: what influences (in) effectiveness? ”, Journal of Change Management , Vol.  20 No.  4 , pp.  369 - 386 , doi: 10.1080/14697017.2020.1758748 .

Guest , D. , Sanders , K. , Rodrigues , R. and Oliveira , T. ( 2021 ), “ Signalling theory as a framework for analysing human resource management processes and integrating human resource attribution theories: a conceptual analysis and empirical exploration ”, Human Resource Management Journal , Vol.  31 No.  3 , pp.  796 - 818 , doi: 10.1111/1748-8583.12326 .

Hannah , S. , Schaubroeck , J. and Peng , A. ( 2016 ), “ Transforming followers' value internalization and role self-efficacy: dual processes promoting performance and peer norm-enforcement ”, Journal of Applied Psychology , Vol.  101 No.  2 , pp.  252 - 266 , doi: 10.1037/apl0000038 .

Hussain , T. , Iren , P. and Rice , J. ( 2019 ), “ Determinants of innovative behaviors among self-initiated expatriates ”, Personnel Review , Vol.  49 No.  2 , pp.  349 - 369 , doi: 10.1108/PR-09-2018-0321 .

Jia , J. , Yan , J. , Jahanshahi , A. , Lin , W. and Bhattacharjee , A. ( 2020 ), “ What makes employees more proactive? Roles of job embeddedness, the perceived strength of the HRM system and empowering leadership ”, Asia Pacific Journal of Human Resources , Vol.  58 No.  1 , pp.  107 - 127 , doi: 10.1111/1744-7941.12249 .

Kou , X. , Kurdi-Nakra , H. and Pak , J. ( 2022 ), “ The framework of first-line manager's HR role identity: a Multi-actor HR involvement perspective ”, Human Resource Management Review , Vol.  32 No.  4 , 100898 , doi: 10.1016/j.hrmr.2022.100898 .

Kubiak , E. ( 2022 ), “ Increasing perceived work meaningfulness by implementing psychological need-satisfying performance management practices ”, Human Resource Management Review , Vol.  32 No.  3 , 100792 , doi: 10.1016/j.hrmr.2020.100792 .

Lee , Q. , Townsend , K. and Wilkinson , A. ( 2020 ), “ Frontline managers' implementation of the formal and informal performance management systems ”, Personnel Review , Vol.  50 No.  1 , pp.  379 - 398 , doi: 10.1108/PR-11-2019-0639 .

Leroy , H. , Segers , J. , Van Dierendonck , D. and Den Hartog , D. ( 2018 ), “ Managing people in organizations: integrating the study of HRM and leadership ”, Human Resource Management Review , Vol.  28 No.  3 , pp.  249 - 257 , doi: 10.1016/j.hrmr.2018.02.002 .

Ligon , G. , Graham , K. , Edwards , A. , Osburn , H. and Hunter , S. ( 2012 ), “ Performance management: appraising performance, providing feedback, and developing for creativity ”, in Muford , M. (Ed.), Handbook of Organizational Creativity , Academic Press , Cambridge, MA , pp.  633 - 666 , doi: 10.1016/B978-0-12-374714-3.00025-2 .

Meier‐Barthold , M. , Biemann , T. and Alfes , K. ( 2023 ), “ Strong signals in HR management: how the configuration and strength of an HR system explain the variability in HR attributions ”, Human Resource Management , Vol.  62 No.  2 , pp. 229 - 246 , doi: 10.1002/hrm.22146 .

Mertens , S. , Schollaert , E. and Anseel , F. ( 2021 ), “ How much feedback do employees need? A field study of absolute feedback frequency reports and performance ”, International Journal of Selection and Assessment , Vol.  29 Nos 3-4 , pp.  326 - 335 , doi: 10.1111/ijsa.12352 .

Murphy , K. ( 2020 ), “ Performance evaluation will not die, but it should ”, Human Resource Management Journal , Vol.  30 No.  1 , pp.  13 - 31 , doi: 10.1111/1748-8583.12259 .

Mustafa , M. , Coetzer , A. , Ramos , H. and Fuhrer , J. ( 2021 ), “ Exploring the effects of small-and medium-sized enterprise employees' job satisfaction on their innovative work behaviours: the moderating effects of personality ”, Journal of Organizational Effectiveness , Vol.  8 No.  2 , pp.  228 - 250 , doi: 10.1108/JOEPP-07-2020-0133 .

Nishii , L. and Paluch , R. ( 2018 ), “ Leaders as HR sensegivers: four HR implementation behaviors that create strong HR systems ”, Human Resource Management Review , Vol.  28 No.  3 , pp.  319 - 323 , doi: 10.1016/j.hrmr.2018.02.007 .

Ostroff , C. and Bowen , D. ( 2016 ), “ Reflections on the 2014 decade award: is there strength in the construct of HR system strength? ”, Academy of Management Review , Vol.  41 No.  2 , pp.  196 - 214 , doi: 10.5465/amr.2015.0323 .

Parker , S. , Bindl , U. and Strauss , K. ( 2010 ), “ Making things happen: a model of proactive motivation ”, Journal of Management , Vol.  36 No.  4 , pp.  827 - 856 , doi: 10.1177/2F0149206310363732 .

Pereira , C. and Gomes , J. ( 2012 ), “ The strength of human resource practices and transformational leadership: impact on organisational performance ”, International Journal of Human Resource Management , Vol.  23 No.  20 , pp.  4301 - 4318 , doi: 10.1080/09585192.2012.667434 .

Presbitero , A. , Teng-Calleja , M. and Farndale , E. ( 2022 ), “ Employee perceptions of HRM system strength: examining outcome and boundary conditions among HR and non-HR employees ”, Personnel Review , Vol.  51 No.  9 , pp. 2142 - 2161 , doi: 10.1108/PR-12-2020-0878 .

Prieto , I. and Pérez-Santana , M. ( 2014 ), “ Managing innovative work behavior: the role of human resource practices ”, Personnel Review , Vol.  43 No.  2 , pp.  184 - 208 , doi: 10.1108/PR-11-2012-0199 .

Sanders , K. , Jorgensen , F. , Shipton , H. , Van Rossenberg , Y. , Cunha , R. , Xiaobei , L. , Wong , S. and Dysvik , A. ( 2018 ), “ Performance‐based rewards and innovative behaviors ”, Human Resource Management , Vol.  57 No.  6 , pp. 1455 - 1468 , doi: 10.1002/hrm.21918 .

Sanders , K. and Yang , H. ( 2016 ), “ The HRM process approach: the influence of employees' attribution to explain the HRM‐performance relationship ”, Human Resource Management , Vol.  55 No.  2 , pp.  201 - 217 , doi: 10.1002/hrm.21661 .

Sanders , K. , Bednall , T. and Yang , H. ( 2021 ), “ HR strength: past, current and future research ”, in  Sanders , K. , Yang , H. and Patel , C. (Eds), Handbook on HR Process Research , Edward Elgar Publishing , pp.  27 - 45 .

Schleicher , D. , Baumann , H. , Sullivan , D. , Levy , P. , Hargrove , D. and Barros-Rivera , B. ( 2018 ), “ Putting the system into performance management systems: a review and agenda for performance management research ”, Journal of Management , Vol.  44 No.  6 , pp.  2209 - 2245 , doi: 10.1177/0149206318755303 .

Singh , N. , Bamel , U. and Vohra , V. ( 2021 ), “ The mediating effect of meaningful work between human resource practices and innovative work behavior: a study of emerging market ”, Employee Relations , Vol.  43 No.  2 , pp.  459 - 478 , doi: 10.1108/ER-04-2020-0150 .

Van Thielen , T. , Decramer , A. , Vanderstraeten , A. and Audenaert , M. ( 2018 ), “ When does performance management foster team effectiveness? A mixed‐method field study on the influence of environmental extremity ”, Journal of Organizational Behavior , Vol.  39 No.  6 , pp.  766 - 782 , doi: 10.1002/job.2297 .

Van Thielen , T. , Decramer , A. , Vanderstraeten , A. and Audenaert , M. ( 2022 ), “ The effects of performance management on relational coordination in policing: the roles of content and process ”, International Journal of Human Resource Management , Vol.  33 No.  7 , pp.  1377 - 1402 , doi: 10.1080/09585192.2020.1779779 .

Van Veldhoven , M. , Dorenbosch , L. , Breugelmans , A. and Van De Voorde , K. ( 2017 ), “ Exploring the relationship between job quality, performance management, and career initiative: a two-level, two-actor study ”, Sage Open , Vol.  7 No.  3 , 2158244017721734 , doi: 10.1177/2158244017721734 .

Van Waeyenberg , T. and Decramer , A. ( 2018 ), “ Line managers' AMO to manage employees' performance: the route to effective and satisfying performance management ”, International Journal of Human Resource Management , Vol.  29 No.  22 , pp.  3093 - 3114 , doi: 10.1080/09585192.2018.1445656 .

Van Waeyenberg , T. , Peccei , R. and Decramer , A. ( 2022 ), “ Performance management and teacher performance: the role of affective organizational commitment and exhaustion ”, International Journal of Human Resource Management , Vol.  33 No.  4 , pp.  623 - 646 , doi: 10.1080/09585192.2020.1754881 .

van Woerkom , M. and Kroon , B. ( 2020 ), “ The effect of strengths-based performance appraisal on perceived supervisor support and the motivation to improve performance ”, Frontiers in Psychology , Vol.  11 No.  2020 , p. 1883 , doi: 10.3389/fpsyg.2020.01883 .

Yan , J. and Hu , W. ( 2022 ), “ Environmentally specific transformational leadership and green product development performance: the role of a green HRM system ”, International Journal of Manpower , Vol.  43 No.  3 , pp.  639 - 659 , doi: 10.1108/IJM-05-2020-0223 .

Acknowledgements

This study was supported by a grant from the Ghent University Special Research Fund (BOF) [BOF.STA.2015.0032.01-BOF15/STA/049].

Corresponding author

About the authors.

Robin Bauwens* is assistant professor at Tilburg University, Department of Human Resource Studies (Netherlands). His research is situated at the crossroads of leadership, HRM and the digital transformation of work. Contact: Department of Human Resource Studies, Tilburg University P.O. Box 90153, 5000 LE Tilburg, The Netherlands.

Mieke Audenaert is associate professor at Ghent University, Faculty of Economics and Business Administration (Belgium). Her research focuses on people management, HRM and leadership in public, non-profit and social profit organizations. Contact: Department of Marketing, Innovation, and Organization, Ghent University Tweekerkenstraat 2, 9000 Gent, Belgium.

Adelien Decramer is associate professor at Ghent University, Faculty of Economics and Business Administration (Belgium). Her research focuses on performance management, organizational behavior and HRM in the public, non-profit and social profit sector. Contact: Department of Marketing, Innovation, and Organization, Ghent University Tweekerkenstraat 2, 9000 Gent, Belgium.

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Performance Management System. A Literature Review

Cite this chapter

research paper on performance management system

Part of the book series: Contributions to Management Science ((MANAGEMENT SC.))

4996 Accesses

2 Citations

This Chapter proposes a broad systematic review of PMS design, describing the evolution of the approaches to PMS design, based on the application of theories; introducing both concepts and frameworks that characterise the field and clearly call out for more research on a comprehensive PMS framework; and showing how PMS mechanisms should relate to each other in order to develop both efficiency and innovation, which result in long-term survival. From the review on PMS design, we can argue that effective design of PMS design is contingent to both external and internal variables; financial performance measures are more and more assessed together with non-financial performance measures; the link between PMS and strategy should be enacted trough different kind of PM mechanisms; PMS is a dynamic package of PM mechanisms, which should be considered as a whole in order to assess the overall effectiveness. Finally, since the analysis of the effect of single mechanisms on the overall effectiveness is partial and problematic, there is a call for more loosely coupled PMSs, which develop both control and flexibility.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

The transition from measurement to management of performance has been called the second wave of knowledge management, since in the first wave “knowledge management – in particular in Nonaka’s view – concerns the single individual’s personal tacit knowledge and the subsequent problem of distributing such knowledge to other individuals in the organisation”, while in the second wave “knowledge management is about management control where managers combine, apply and develop a corporate body of knowledge resources to produce and use value around the company’s services” (Mouritsen and Larsen 2005 : 388).

Anne Huff defined the systematic literature review as the “explicit procedures to identify, select, and critically appraise research relevant to a clearly formulated question” (Huff 2009 : 148).

Although the review is focused on ‘performance management’ and ‘performance management system’, the search terms included other concepts, which are closely related to the main research question.

The sophistication of the management accounting systems has been defined as the “capability of an MAS to provide a broad spectrum of information relevant for planning, controlling, and decision-making all in the aim of creating or enhancing value” (Abdel-Kader and Luther 2008 : 3).

Previous studies on leadership style analysed the effect of this variable on budgetary participation, and the results were statistically significant (Brownell 1983 ).

Tolerance for ambiguity measures “the extent to which one feels threatened by ambiguity or ambiguous situations” (Chong 1998 : 332).

TCE develops the idea that controlling complex economic transactions by “hard contracting” is expensive and an optimal choice between firm and market governance should be taken according to asset specificity. “If assets are non-specific, markets enjoy advantages in both production cost and governance cost respects […]. As assets become more specific, however, the aggregation benefits of markets […] are reduced and exchange takes on a progressively stronger bilateral character” (Williamson 1981 : 558).

Even though the first framework developed four perspectives (financial, internal business, customer, and innovation and improvement), Kaplan and Norton specified that each firm, or unit, using the BSC should adjust the number and focus of perspectives and their measures to the specific case under analysis. Therefore, the number of perspectives can be higher than four and the perspectives caption can be changed according to the strategic issues that the firm has to monitor in order to be successful.

Together with the BSC, other performance measurement systems based on both financial and non-financial performance measures have been developed, such as the Results and Determinants (Fitzgerald et al. 1991 ), the Performance Pyramid (Lynch and Cross 1995 ), and the PISCI (Azofra et al. 2003 ).

According to Kim and Oh, the performance measures related to R&D departments should be based on behavioural and qualitative measures, such as “leadership and mentoring for younger researchers”, and appraised by a “bottom up (e.g., R&D researchers’ evaluation of their own bosses say, R&D managers) as well as horizontal (e.g., peers and/or colleagues)” evaluation scheme (Kim and Oh 2002 : 19).

Simons described the old management control philosophy as a “command-and-control” one, in which strategy setting follows a top-down direction, a lot of emphasis is put on standardization and efficiency, results are compared to and should be aligned to plan, and much effort is devoted to keeping things on track and minimizing the number of “surprises”. On the other hand, he pointed out that the new management control philosophy is more concerned with “creativity […], new organizational forms, […] the importance of knowledge as a competitive asset”, which has resulted in “market-driven strategy, customization, continuous improvement, meeting customer needs, and empowerment” (Simons 1995 : 3).

Mission statement, vision and corporate credo are all examples of “organizational definitions”.

However, Simons also warned about setting boundaries that could inhibit adaptive change and survival ( 1995 : 55–53).

Benefits from managerial creativity relate to all the new alternatives and solutions that managers can invent in trying to either create value for the organization or solve problems (Christenson 1983 ; Nelson and Winter 1982 ), while dysfunctionalities refer to research activities that are either too risky or too vague, and thus not value creating.

Argyris and Schon also called the intended strategy an “espoused theory” in contrast to “theory-in-use” (Argyris and Schon 1978 : 10–11).

Simons argued that critical performance variables are “those factors that must be achieved or implemented successfully for the intended strategy of the business to succeed” (p. 63); they can be identified through effectiveness and efficiency criteria (Anthony 1965 ). He also agreed with Lawler and Rhode ( 1976 ) that critical performance variables should be related to objective, rather than subjective measures; complete, instead of incomplete; and responsive, rather than unresponsive, measures. Simons also posited that all the three features rarely occur in diagnostic control systems (Simons 1995 : 76).

Simons asserted that in “normal competitive conditions, senior managers with a clear sense of strategic vision choose very few – usually only one – management control system at any point in time” (Simons 1991 ). The reasons for this limited choice are related to both economic and cognitive, as well as strategic issues. Since the interactive use of control systems require managerial attention, managers will be distracted by other day-to-day operations, which can be handled only for one system at a time. From a cognitive perspective, individuals can cope and make decisions simultaneously only with a limited amount of information; otherwise they will be overwhelmed by data. From a strategic standpoint, “the primary reason for using a control system interactively is to activate learning and experimentation” (Simons 1995 : 116); therefore it is better to avoid poor analysis, or decision paralysis coming from too many projects under analysis.

Nonetheless, Collier acknowledges the implementation of the beliefs system lever of control (Collier 2005 ).

She also stressed that investigating “how differences in interpretation of strategic contingencies shape management control systems would enrich Simons’ model” (Gray 1990 : 146).

The portfolio of management control mechanisms is made up of “standard operating procedures, position descriptions, personal supervision, budgets, performance measurement, reward systems and internal governance, and accountability arrangements [as well as …] less obtrusive forms of control, such as personnel selection, training and socialization processes” (Abernethy and Chua 1996 : 573).

An example of such frameworks is the value based management tool introduced by Ittner and Larcker ( 2001 ).

Mission has been defined as the “overriding purpose of the organization in line with the values or expectations of stakeholders”, while the vision develops the “desired future state: the aspiration of the organization” (Johnson et al. 2005 : 13).

In their work, Malmi and Brown specified that, although their framework represents a broad typology, it is also a parsimonious one, since it encompasses only five types of control (Malmi and Brown 2008 : 291).

Merchant and Van der Stede’s framework develops different forms of control according to the different objects under control, which are culture, personnel, action and results controls (Merchant and Van der Stede 2007 ).

Nonetheless, the authors acknowledged that culture may sometimes be beyond managerial control.

On the issue of a tentative framework, the authors call for “further research [that] should reveal the missing and unnecessary elements in it” (Malmi and Brown 2008 : 295).

Although budgets can cover a shorter, or longer period, it is usually based on a 12-month period.

Giorgio Brunetti stressed that both purposes should be accomplished by the management control system, although one of the two may be “stressed” (Brunetti 1979 : 69) a little bit further, indeed, he argued that a control system, which is uncoupled from the rewarding system, results in an amount of information aimed at sustaining, rather that coordinating, operations (p. 70).

In line with the contingency approach, the effective design, according to Brunetti, lies in the “congruency”, or “fit” of management control system’s variables with both management control system’s inputs and outputs (Brunetti 1979 : 98).

Other limitations to the cybernetic approach to the design of management control system can be found elsewhere in this work (§ 2.3).

To Mella, a system of transformation is an “‘entity’ able to transform certain ‘objects’ that enter the system into different ‘objects’ which leave the system” (Mella 1992 : 456).

Abdel-Kader M, Luther R (2008) The impact of firm characteristics on management accounting practices: a UK-based empirical analysis. Br Account Rev 40:2–27

Google Scholar  

Abernethy MA, Brownell P (1997) Management control systems in research and development organizations: the role of accounting, behavior and personnel controls. Account Organ Soc 22(3–4):233–248

Abernethy MA, Brownell P (1999) The role of budgets in organizations facing strategic change: an exploratory study. Account Organ Soc 24(3):189–205

Abernethy MA, Chua WF (1996) A field study of control system “Redesign”: the impact of institutional processes on strategic choice. Contemp Account Res 13(2):569–606

Abernethy MA, Bouwens J, van Lent L (2010) Leadership and control system design. Manag Account Res 21:2–16

Ackoff RL (1977) National development planning revisited. Oper Res 25:207–218

Amigoni F (1995) Misurazioni d’azienda. Programmazione e controllo, (a cura di). Giuffré, Milano

Anthony RN (1965) Planning and control systems: a framework for analysis. Harvard Business School Division of Research, Boston

Argyris C, Schon D (1978) Organisational learning: a theory of action perspective. Addison Wesley, Reading

Azofra V, Prieto B, Santidrián A (2003) The usefulness of a performance measurement system in the daily life of an organization: a note on a case study. Br Account Rev 35:367–384

Beekun RI, Glick WH (2001) Organization structure from a loose coupling perspective: a multidimensional approach. Decis Sci 32(2):227–250

Beer SA (1979) The heart of enterprise. Wiley, London/New York

Beer SA (1981) Brain of the firm; second edition. Wiley, London/New York

Berry AJ, Coad AF, Harris EP, Otley DT, Stringer C (2009) Emerging themes in management control: a review of recent literature. Br Account Rev 41:2–20

Bisbe J, Otley D (2004) The effects of an interactive use of control systems on product innovation. Account Organ Soc 29:709–737

Bisbe J, Batista-Foguet J-M, Chenhall R (2007) Defining management accounting constructs: a methodological note on the risks of conceptual misspecification. Account Organ Soc 32:789–820

Brignall S, Ballantine J (2004) Strategic enterprise management systems: new directions from research. Manag Account Res 15(2):225–240

Broadbent J, Laughlin R (2009) Performance management systems: a conceptual model. Manag Account Res 20:283–295

Brownell P (1983) The motivational impact of management-by-exception in a budgetary context. J Account Res 21:456–472

Brunetti G (1979) Il controllo di gestione in condizioni ambientali perturbate. FrancoAngeli, Milano

Brusoni S, Prencipe A, Pavitt K (2001) Knowledge specialisation, organizational coupling and the boundaries of the firm: why firms know more than they make? Adm Sci Q 46(4):597–621

Burns T, Stalker GM (1961) The management of innovation. Tavistock, London

Chandler AD Jr (1962) Strategy and structure: chapters in the history of the American industrial enterprise. MIT Press, Cambridge, MA

Chenhall RH (2003) Management control system design within its organizational context: findings from contingency-based research and directions for the future. Account Organ Soc 28(2–3):127–168

Chenhall R (2005) Integrative strategic performance measurement systems, strategic alignment of manufacturing, learning and strategic outcomes: an exploratory study. Account Organ Soc 30(5):395–422

Chenhall RH (2008) Accounting for the horizontal organization: a review essay. Account Organ Soc 33(4–5):517–550

Choe J, Langfield-Smith K (2004) The effects of national culture on the design of management accounting information systems. J Comp Int Manage 7(1)

Choe JM (1998) The effects of user participation on the design of accounting information systems. Inf Manage 34(3):185–198

Chong VK (1998) Testing the contingency ‘fit’ on the relation between management accounting systems and managerial performance: a research note on the moderating role of tolerance for ambiguity. Br Account Rev 30:331–342

Christenson C (1983) The methodology of positive accounting. Account Rev 53(1):1–22

Collier P (2005) Entrepreneurial control and the construction of a relevant accounting. Manag Account Res 16:321–339

Daft RL (1978) A dual-core model of organizational innovation. Acad Manag J 21(2):193–210

Davila A (2000) An empirical study on the drivers of management control systems’ design in new product development. Account Organ Soc 25(4–5):383–409

Dossi A, Patelli L (2008) The decision-influencing use of performance measurement systems (PMS) in relationships between headquarters and subsidiaries. Manag Account Res 19(1):126–148

Doty HD, Glick WH, Huber GP (1993) Fit, equifinality, and organizational effectiveness: a test of two configurational theories. Acad Manag J 36:1196–1250

Dubois A, Gadde L-E (2002) Systematic combining: an abductive approach to case research. J Bus Res 55(7):553–560

Duff A (1996) The literature search: a library-based model for information skills instruction. Libr Rev 45(4):14–18

Ferreira A, Otley D (2005) The design and use of management control systems: an extended framework for analysis. Social Science Research Network. http://papers.ssrn.com/sol3/papers.cfm?abstract_id¼682984

Ferreira A, Otley D (2009) The design and use of performance management systems: an extended framework for analysis. Manag Account Res 20:263–282

Fitzgerald L, Johnston R, Brignall TJ, Silvestro R, Voss C (1991) Performance measurement in service businesses. The Chartered Institute of Management Accountants, London

Galbraith JK (1973) Designing complex organizations. Addison-Wesley, Reading

Gerdin J (2005) Management accounting system design in manufacturing departments: an empirical investigation using a multiple contingencies approach. Account Organ Soc 30:99–126

Gietzmann MB (1996) Incomplete contracts and the make or buy decision: governance design and attainable flexibility. Account Organ Soc 21(6):611–626

Gray RH (1990) The greening of accountancy: the profession after pearce. ACCA, London

Green SG, Welsh MA (1988) Cybernetics and dependence: reframing the control concept. Acad Manag Rev 13(2):287–301

Gresov C, Drazin R (1997) Equifinality: functional equivalence in organization design. Acad Manag Rev 22(2):403–428

Gupta AK, Govindarajan V (1985) Linking control systems to business unit strategy: impact on performance. Account Organ Soc 10(1):51–66

Harrison GL (1993) Reliance on accounting performance measures in superior evaluative style. The influence of national culture and personality. Account Organ Soc 18:319–339

Harrison G, McKinnon J (1999) Cross-cultural research in management control systems design: a review of the current state. Account Organ Soc 24:483–506

Heider F (1959) The psychology of interpersonal relations. Wiley, New York

Henry J-F (2006) Management control systems and strategy: a resource-based perspective. Account Organ Soc 31:529–558

Huff AS (2009) Designing research for publication. Sage, Thousand Oaks

Ittner CD, Larcker DF (2001) Assessing empirical research in managerial accounting: a value-based management perspective. J Account Econ 32:349–410

Ittner CD, Larcker DF, Randall T (2003) Performance implications of strategic performance measurement in financial services firms. Account Organ Soc 28:715–741

Jensen M, Meckling W (1976) Theory of the firm: managerial behavior, agency costs, and ownership structure. J Financ Econ 3:305–360

Johnson G, Scholes K, Whittington R (2005) Exploring corporate strategy. FT Prentice-Hall, London

Johnson HT, Kaplan RS (1987) Relevance lost: the rise and fall of management accounting. Harvard Business School Press, Boston

Kald M, Nilsson F, Rapp B (2000) On the strategy and management control: the importance of classifying the strategy of the business. Br J Manag 11:197–212

Kaplan RS, Norton DP (1992) The balanced scorecard- measures that drive performance. Harv Bus Rev 70(1):71–79

Kaplan RS, Norton DP (1993) Putting the balanced scorecard to work. Harv Bus Rev 71(5):134–147

Kaplan RS, Norton DP (2000) Having trouble with your strategy? Then map it. Harv Bus Rev 78:167–176

Kaplan RS, Norton DP (2007) Using the balanced scorecard as a strategic management system. Harv Bus Rev 85(7/8):172–180

Kim B, Oh H (2002) An effective R&D performance measurement system: survey of Korean R&D researchers. Omega 30(1):19–31

Kominis G, Emmanuel CR (2007) The expectancy-valence theory revisited: developing an extended model of managerial motivation. Manag Account Res 18:49–75

Langfield-Smith K (1997) Management control systems and strategy: a critical review. Account Organ Soc 22:207–232

Langfield-Smith K (2006) Management accounting: information for managing and creating value. McGraw-Hill, Sydney

Langfield-Smith K (2008) Strategic management accounting: how far have we come in 25 years? Account Audit Account J 21(2):204–228

Langfield-Smith K, Smith D (2003) Management control systems and trust in outsourcing relationships. Manag Account Res 14:281–307

Lawler EE, Porter LW (1967) The effects of performance on job satisfaction. Ind Relat 7:20–28

Lawler EE, Rhode JG (1976) Information and control in organizations. Goodyear, Pacific Palisades

Lawrence PR, Dyer D (1983) Renewing American industry. Free Press, New York

Lax DA, Sebenius JK (1986) The mnager as negotiator: bargaining for cooperation and competitive gain. Free Press, New York

Li P, Tang G (2009) Performance measurement design within its organisational context—evidence from China. Manag Account Res 20:193–207

Lynch RL, Cross KF (1995) Measure up!: how to measure corporate performance. Blackwell, Cambridge, MA

Macintosh NB, Daft RL (1987) Management control systems and departmental interdependencies: an empirical study. Account Organ Soc 12(1):49–61

Mahama H (2006) Management control systems, cooperation and performance in strategic supply relationships: a survey in the mines. Manag Account Res 17:315–339

Malina M, Selto F (2001) Controlling and communicating strategy: an empirical test of the effectiveness of the balanced scorecard. J Manag Account Res 13:47–90

Malina M, Selto F (2004) Choice and change of measures in performance measurement models. Manag Account Res 15(4):441–469

Malmi T, Brown DA (2008) Management control systems as a package – opportunities, challenges and research directions. Manag Account Res 19:287–300

Mella P (1992) Economia aziendale. UTET, Torino

Mella P (1997) Controllo di gestione. UTET, Torino

Mella P (2005) Performance indicators in business value-creating organizations. Econ Aziendale Online 2(2005):25–52

Mella P, Pellicelli M (2008) The origin of value based management: five interpretative models of an unavoidable evolution. Int J Knowl Cult Change Manag 8(2):23–32

Merchant K (1998) Modern management control systems. Upper Saddle River, Prentice Hall

Merchant KA, Otley DT (2007) A review of the literature on control and accountability. In: Chapman CS, Hopwood AG, Shields MD (eds) Handbook of management accounting research. Elsevier, Amsterdam, pp 785–804

Merchant KA, Van der Stede W (2007) Management control systems: performance measurement, evaluation and incentives. Financial Times Press, Harlow

Miles MB, Sullivan E, Gold BA, Taylor BL, Sieber SD, Wilder DE (1978) Designing and starting innova-tive schools: a field study of social architecture in education. Final report. Center for Policy Research, New York

Mouritsen J, Larsen HT (2005) The 2nd wave of knowledge management: the management control of knowledge resources through intellectual capital information. Manag Account Res 16(3):371–394

Nanni AJ, Dixon JR, Vollmann TE (1992) Integrated performance measurement: management accounting to support the new manufacturing realities. J Manag Account Res 4(Fall):1–19

Neely A (2008) Does the balanced scorecard work: an empirical investigation. Research paper series, no. 1/08. Available online at www.som.cranfield.ac.uk/som/research/researchpapers.asp

Nelson RR, Winter S (1982) An evolutionary theory of economic change. The Belknap Press of Harvard University, London

Nilsson F (2000) Parenting styles and value creation: a management control approach. Manag Account Res 11:89–112

Nilsson F (2002) Strategy and management control systems: a study of the design and use of management control systems following takeover. Account Finance 42(1):41–71

Nilsson F, Kald M (2002) Recent advances in performance management: the Nordic case. Eur Manag Rev 20(3):235–245

Norreklit H (2000) The balance on the balanced scorecard: a critical analysis of some of its assumptions. Manag Account Res 11:65–88

Orton JD, Weick KE (1990) Loosely coupled systems: a reconceptualization. Acad Manag Rev 15(2):203–233

Otley DT (1978) Budgetary use and managerial performance. J Account Res 16:122–149

Otley DT (1980) The contingency theory of management accounting: achievement and prognosis. Account Organ Soc 5:413–428

Otley DT (1999) Performance management: a framework for management control systems research. Manag Account Res 10:363–382

Otley DT (2008) Did Kaplan and Johnson get it right? Account Audit Account J 21(2):229–239

Otley D, Berry A (1980) Control, organisation and accounting. Account Organ Soc 5(2):231–244

Ouchi WG (1979) A conceptual framework for the design of organizational control mechanisms. Manag Sci 25:833–848

Perego PM, Hartmann FGH (2009) Aligning performance measurement systems with strategy: the case of environmental strategy. Abacus 45(4):397–428

Perrow C (1970) Organizational analysis: a sociological view. Tavistock Publications, London

Prahalad CK, Bettis RA (1986) The dominant logic: a new linkage between diversity and performance. Strateg Manag J 7(6):485–501

Sandelin M (2008) Operation of management control practices as a package – a case study on control system variety in a growth firm context. Manag Account Res 19:324–343

Senge PM (1990) The fifth discipline: the art and practice of the learning organization. Doubleday Currency, New York

Simons R (1991) Strategic orientation and top management attention to control systems. Strateg Manag J 12:49–62

Simons R (1995) Levers of control: how managers use innovative control systems to drive strategic renewal. Harvard Business School Press, Boston

Spekle RF (2001) Explaining management control structure variety: a transaction cost economics perspective. Account Organ Soc 26(4–5):419–441

Stringer ET (2007) Action research, 3rd edn. Sage, London

Sundbo J, Gallouj F (2000) Innovation as a loosely coupled system in services. Int J Serv Technol Manag 1(1):15–36

Tuomela T (2005) The interplay of different levers of control: a case study of introducing a new performance measurement system. Manag Account Res 16(3):293–320

Weick KE (1976) Education systems as loosely coupled systems. Adm Sci Q 21:1–19

Widener SK (2007) An empirical analysis of the levers of control framework. Account Organ Soc 32(7):757–788

Williamson OE (1975) Markets and hierarchies. Free Press, New York

Williamson OE (1979) Transaction cost economics: the governance of contractual relations. J Law Econ 22:233–261

Williamson OE (1981) The economics of organization: the transaction cost approach. Am J Sociol 87(3):548–577

Woodward J (1958) Industrial organization. Theory and Practice, London

Zappa G (1927) Tendenze nuove negli studi di ragioneria. IES, Milano

Download references

Author information

Authors and affiliations.

Department of Economics and Management, University of Pavia, Pavia, Italy

Chiara Demartini

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Demartini, C. (2014). Performance Management System. A Literature Review. In: Performance Management Systems. Contributions to Management Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36684-0_3

Download citation

DOI : https://doi.org/10.1007/978-3-642-36684-0_3

Published : 20 June 2013

Publisher Name : Springer, Berlin, Heidelberg

Print ISBN : 978-3-642-36683-3

Online ISBN : 978-3-642-36684-0

eBook Packages : Business and Economics Business and Management (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Policies and ethics

Go to homepage

Performance management: Meaning, stages, and best practices

by Aleksandra Masionis

Updated on August 9, 2024

Develop performance management systems

Create a culture that means business™

Email address

Managing employee performance ensures that your workforce remains productive, engaged, and motivated. By setting clear expectations, providing regular feedback, and recognizing achievements, organizations can create a culture of high performance that drives business success. But without balancing the need for honest, critical assessments with the need to maintain positive employee morale — and ensuring evaluations are consistent and fair — employees may resist performance management initiatives, perceiving them as punitive, overly bureaucratic, and biased.

The right performance management strategy and tools can address these challenges and transform the process into a positive experience for both managers and employees. Let’s explore what great performance management entails, taking a look at each part of the performance management cycle and how your company can improve productivity and engagement across your entire workforce.

Performance management is the process of assessing and improving employee performance. It revolves around setting clear expectations, providing regular feedback, and supporting employee development. When done right, performance management ensures that team members understand what your company expects of them and how their roles contribute to your organization’s success. By establishing a structured process for evaluating and developing performance, organizations can identify their most productive employees and support team members as they grow.

5 stages of the performance management cycle

The performance management cycle consists of five key stages, each of which plays an important role in enhancing employee productivity.

1. Create a performance plan

The planning stage involves setting measurable goals for employees, informed by their professional interests and company goals. Managers and employees should collaborate to define expectations, establish performance criteria, and outline the steps needed to achieve set goals. This cooperative approach helps build a common understanding of employees’ roles and responsibilities, while making it more likely that team members will buy into the performance review process. Managers should also empower employees by outlining the resources, training, and other support needed to meet objectives in the performance plan itself — and then provide everything they’ve promised, so team members can reach their full potential.

2. Monitor employee performance

After developing a plan, managers should monitor employee performance as they make progress towards the established goals. During this stage, leaders should gather data to concretely assess how well employees are performing — as well as how they feel about the performance management process and other key aspects of your company. Managers should also hold regular check-ins with direct reports to discuss performance and provide a channel for honest, two-way feedback .

By maintaining open lines of communication, manager will never lack an opportunity to motivate team members by offering words of encouragement and recognition . They’ll also be able to address issues or challenges that may arise in a timely fashion and provide any guidance needed to ensure employees stay on the track to success.

3. Help employees develop

Explaining what you want team members to do and telling them to “go at it” won’t result in the performance your organization is looking for. Instead, managers should work with employees to build personalized development plans that address specific skill gaps and align with team members’ career aspirations. Identify professional development needs through performance assessments, and then provide training opportunities targeted to meet those needs while matching employees’ individual learning styles. These development initiatives can include coaching initiatives , on-the-job training, and access to online learning resources.

4. Review employee performance

Conducting reviews is a core part of any performance management process. These assessments are an opportunity for managers and employees to reflect on achievements and ensure that both the organization’s and team member’s needs are being met. Managers must evaluate how well employees have met their goals and provide comprehensive feedback — highlighting the positives while discussing areas for improvement. Leaders should base each review on objective data and observations, staying alert for any biases that may impact their judgment.

As noted above, annual performance reviews aren’t sufficient. By the time they occur, much of the feedback from both sides’ will be largely irrelevant, unactionable, and, by extension, mutually frustrating to receive. Setting a relatively frequent cadence of performance-focused, one-on-one meetings with direct reports, supplementing these discussions with more formal reviews held quarterly or twice a year, and establishing a range of open feedback channels will ensure that both employees and managers can adjust their approach before issues become intractable.

Your company can facilitate this environment of continuous improvement by adopting performance management software that allows managers and employees to share feedback instantly, set goals, track progress, and document performance discussions. And don’t forget to adopt an employee engagement platform that lets employees provide feedback anonymously at any time. Otherwise, they’re likely to hold back some of their real thoughts they aren’t comfortable sharing with management directly, and your business will miss out on key ways to improve performance management and the employee experience .

5. Recognize and reward employees

Recognizing employees for their contributions , both large and small, is perhaps the most essential part of performance management — and certainly the one that should be most enjoyable for all parties involved. Recognition can take many forms, from a quick email of thanks, to a well-deserved bonus, to public appreciation during an all-hands meeting. But to make a real impact, your organization needs to streamline and scale its recognition efforts , so every team member can provide meaningful appreciation with the push of a button, no matter where they find themselves. The fastest way to accomplish this is with an employee recognition solution that brings everyone together in a communal space, where they can show thanks and react to others’ moments of appreciation.

Effective performance management requires a balance between positive reinforcement and constructive feedback — between helping employees meet their own professional goals and empowering them with the resources and guidance needed to achieve those of your business. Achieving this ideal starts with establishing clear standards to measure employee performance against. There’s no need to reinvent the wheel when it comes to setting tangible objectives, though. Instead, use a popular, easy-to-adopt framework like SMART goals or objectives and key results (OKRs) , both of which facilitate clear communication and accountability on the part of managers and employees.

SMART goals are specific, measurable, achievable, relevant, and time-bound. This framework helps employees understand exactly what is expected of them, how their performance will be measured, and the timeline for achieving their objectives. SMART goals provide a roadmap that team members can follow when prioritizing their tasks, and one that managers can use to track employee progress and provide targeted, relevant feedback.

OKRs involve first setting high level objectives and then defining measurable key results that indicate progress toward those goals. They encourage employees to set ambitious goals and keep their focus on outcomes rather than activities. By tracking and regularly reviewing OKRs, organizations gain a valuable, objective yardstick for measuring employee performance while maintaining alignment between individual contributions and shifting business priorities.

There are many aspects of performance management that can benefit from the right HR technology. First, ensure you’ve taken care of the basics. Look for a performance management solution that lets managers and employees set clear, measurable goals, track progress towards them, and easily conduct frequent reviews to maintain alignment and fast track employee development. It should also provide powerful analytics and reporting capabilities that help HR professionals and leaders gain insights into performance trends, identify high performers, and pinpoint areas for individual improvement.

Your company should also look for an employee engagement solution that provides a real time stream of honest employee feedback. The best engagement platforms offer a range of intuitively designed feedback channels, from focused pulse surveys to intelligent chatbots that are able to prompt employees for input. Leaders and HR professionals can then use built-in reporting capabilities and dashboards to see where their performance management system is working and where employees think it could use some improvement.

Last but not least, adopt an employee recognition and rewards platform that allows for both peer-to-peer and manager-to-employee recognition. Beyond the fundamentals, like a centralized place for all team members to provide public recognition, a mobile-friendly app, and integrations with the tools your employees use every day, select a solution that lets team members tie each recognition to meaningful rewards . This isn’t as difficult as it might sound — with a points-based reward system backed by a marketplace filled with millions of options employees actually want, your company can make every performance win truly memorable.

You can take performance management at your company beyond the norm and into the realm of real excellence with the Achievers Employee Experience Platform . It leverages the science of HR to make performance management an engaging and effective process, starting with Achievers Recognize , a comprehensive employee recognition and rewards solution. It provides the incentives needed to keep the performance management cycle running smoothly, with easy-to-use social recognition features and a rewards marketplace filled to bursting with exciting merchandise and experiences.

And thanks to Achievers Listen , your people leaders will never lack real time insights into how employees feel about the performance management process and what they can do to engage them further. By streamlining the collection and analysis of feedback, Listen provides leaders with a deep understanding of what drives team members, so they can make informed decisions that improve employee experience and performance.

See how the Achievers Employee Experience Platform can improve performance management for yourself with a free demo.

Discover the number one reason employees will job hunt in 2024

2024 State of Recognition Report

In this article:

Profile image of author: Aleksandra Masionis

Interested in learning more about Achievers?

We use cookies to help us understand how you use our site so we can show you personalized content and enhance your browsing experience. Learn more by viewing our Privacy Policy

If health organisations and staff engage in research, does healthcare improve? Strengthening the evidence base through systematic reviews

Health Research Policy and Systems volume  22 , Article number:  113 ( 2024 ) Cite this article

56 Accesses

7 Altmetric

Metrics details

There is an often-held assumption that the engagement of clinicians and healthcare organizations in research improves healthcare performance at various levels. Previous reviews found up to 28 studies suggesting a positive association between the engagement of individuals and healthcare organizations in research and improvements in healthcare performance. The current study sought to provide an update.

We updated our existing published systematic review by again addressing the question: Does research engagement (by clinicians and organizations) improve healthcare performance? The search covered the period 1 January 2012 to March 2024, in two phases. First, the formal updated search ran from 1 January 2012 to 31 May 2020, in any healthcare setting or country and focussed on English language publications. In this phase two searches identified 66 901 records. Later, a further check of key journals and citations to identified papers ran from May 2020 to March 2024. In total, 168 papers progressed to full-text appraisal; 62 were identified for inclusion in the update. Then we combined papers from our original and updated reviews.

In the combined review, the literature is dominated by papers from the United States (50/95) and mostly drawn from the Global North. Papers cover various clinical fields, with more on cancer than any other field; 86 of the 95 papers report positive results, of which 70 are purely positive and 16 positive/mixed, meaning there are some negative elements (i.e. aspects where there is a lack of healthcare improvement) in their findings.

Conclusions

The updated review collates a substantial pool of studies, especially when combined with our original review, which are largely positive in terms of the impact of research engagement on processes of care and patient outcomes. Of the potential engagement mechanisms, the review highlights the important role played by research networks. The review also identifies various papers which consider how far there is a “dose effect” from differing amounts of research engagement. Additional lessons come from analyses of equity issues and negative papers. This review provides further evidence of contributions played by systems level research investments such as research networks on processes of care and patient outcomes.

Peer Review reports

There is an often-held assumption that the engagement of clinicians and healthcare organizations in research improves healthcare performance at various levels. This assumption contributed to policy documents from various health organizations promoting research engagement by healthcare providers as a way of improving healthcare, for example, in the United Kingdom [ 1 , 2 , 3 ]. Therefore, it was believed that policy-makers who make relevant decisions, such as on the allocation of resources for health and health research systems, should have access to evidence on the validity of the assumption. In the United Kingdom, two programmes of the National Institute for Health Research (now called the National Institute for Health and Care Research) (NIHR) decided to commission reviews of the global evidence on this [ 1 , 2 , 3 ].

The wide-ranging brief provided for the second review, which was the original review by the authors of this present paper (published in full as Hanney et al. in 2013 [ 3 ] and more succinctly as Boaz et al. in 2015 [ 2 ]), included the additional aim of conducting a theoretically grounded synthesis to explore the mechanisms by which research engagement might improve healthcare [ 3 ]. The protocol for that study considered pertinent global literature, including on accelerating the adoption of evidence in health systems, and ways to enhance the relevance of the research conducted to the needs of health systems. The final protocol published as part of the Hanney et al. report [ 3 ] then used these ideas to identify possible mechanisms that would be worth analysing to help understand the processes that might be at work when research engagement leads to improved health. Among these was the idea that engaging in conducting research increases the ability and willingness of clinicians to use research findings from the global pool of knowledge, and here the concept of “absorptive capacity” was expected to be useful [ 3 ].

Some analyses focussed on the importance of exploring the relationship between research engagement and improved healthcare to contribute towards understanding of the benefits for healthcare performance in the context of a strong research culture. These papers were reviewed in Australia by Harding et al. in 2017 [ 4 ].

As far as we are aware, these three systematic literature reviews published in the second decade of this century provided the first analyses of the empirical evidence available to support the assumption of improved healthcare from research engagement [ 1 , 2 , 3 , 4 ]. Their differing scopes and approaches are summarized briefly in Table  1 .

All three of the reviews reported some evidence of a positive association between research engagement and healthcare performance, but the available evidence was not mature enough to support statements about causality [ 2 ]. Our review [ 2 , 3 ] had the widest scope of the three, reflecting the broad brief given by our NIHR funder. It included an extensive initial mapping exercise, a formal focussed review, and a wider review which drew on the earlier stages to explore, as noted above, the mechanisms by which research engagement might improve healthcare [ 3 ]. Our review identified 33 papers from 9 countries (15 from the United States), 28 of which reported positive findings [ 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ]. Even our review concluded, however, that there did not appear to be a well-structured, steadily accumulating body of knowledge about the benefits associated with research engagement.

In the succeeding years, we have identified a continuing and growing interest in this general topic, therefore an updated review seemed desirable to gather more evidence about how far research engagement might lead to improved healthcare and the mechanisms involved. In addition to these general questions, our original review had identified two specific issues that could usefully be considered further. These were research networks as potentially important mechanisms through which research engagement might improve healthcare, and whether greater amounts of research engagement would have a larger beneficial effect. Our original review also covered some aspects of a third issue (health equity) that has subsequently become increasingly important [ 2 , 3 ].

The growing development of research networks has been associated with efforts to move towards more formalized attempts to boost the role of health research systems in accelerating science and facilitating the translation of research into practice [ 2 , 3 ]. However, at the time of our original review, the evidence was still emerging and its availability was heavily skewed by the different timing of the establishment of formal research networks in different countries.

In the United States, various research networks had been set up in the second half of the last century, and most of the early papers on networks and their role came from there [ 2 ]. These networks in the United States are described in the “Glossary of the United States of America and United Kingdom Research Organizations and Networks Discussed in the Papers” (see Additional file 1 ). They include the National Cancer Institute (NCI)-funded Cancer Community Oncology Program (CCOP), established to encourage outreach and improve equity by bringing the advantages of clinical research to cancer patients in their own communities [ 37 ]. In the United Kingdom in contrast, national research networks were not formally created until this century, too late for any potential benefits to patient outcomes to be fully researched and reported prior to our review which started in 2011. However, even in our original review we were aware of concurrent United Kingdom work to measure those outcomes and to improve patient access to clinical research, and identified a need for further evaluations (see Hanney et al. [ 3 ], pp. 48, 83).

Subsequently, we also became increasingly aware of new studies on the effects of the developing research networks, especially in the United Kingdom, and Boaz et al. identified a promising approach in statistical analysis that could help further analysis [ 2 ]. As set out in the Glossary, there have been policy shifts and organizational changes in the United States and the United Kingdom, and there have been further ones elsewhere, which are designed to promote research networks to address the time lag between the production of research and its use in practice, including various efforts to strengthen links between academic centres and community services. There has also been an increasing emphasis, including within research networks, on the potential research contribution of healthcare professionals other than medical professionals.

Our original review had also noted a partly related second issue as worthy of further attention. This is the question of whether the association between research engagement by healthcare providers and improved healthcare outcomes increases with greater amounts of research participation. There was early evidence that it did. This came, in particular, from the 2008 paper by Majumdar et al. [ 26 ] that compared outcomes for patients with angina in hospitals in the United States having a high level of angina research activity with hospitals with low research activity, and those with no research activity. Other papers compared centres with different levels of research activity within a research network [ 23 ]. However, there was little certainty about extent and implications around this issue at that time, although it has become increasingly important with the development of the comprehensive research networks that we summarize in the Glossary. It also has theoretical implications for the exact nature of the association between research engagement and improved healthcare: in our original review we argued that further data on this effect, and on the time an institution was research active, “are needed to provide evidence of causation” (p. 12) [ 2 ].

These findings also have implications for health equity, the third unresolved issue. More outreach by research networks means more access to clinical research and its benefits for more patients. The United States CCOP has been rightly lauded for achieving this [ 37 ], but can that be squared with the emerging finding that higher levels of research participation in specific provider institutions bring greater benefit to the patients in those centres?

Reflection on these uncertainties further strengthened the argument that with all the developments since our original review, it seemed timely in 2020 to revisit this topic to explore and collate what additional understanding had been gained. While conducting the resulting update, we became aware of some more recent developments. A United Kingdom qualitative systematic review was published in 2021 that explored the impact of research activity by healthcare professionals other than medical professionals [ 38 ], and another UK review published in 2023 focussed on research engagement by allied health professionals (AHPs) [ 39 ]. With few exceptions, the papers specifically on nursing and AHPs in these reviews were typically smaller scale than the papers included in our formal review, and/or usually did not include the quantifiable comparisons that featured in most of our included papers. Nevertheless, these reviews usefully illustrate the growing interest in the contribution of these healthcare professionals in countries such as Australia, Canada and the United Kingdom.

In addition, we identified a large-scale study from the United States by Shahian et al. [ 40 ] that was published in 2022 and examined the link between research engagement and improved healthcare performance in 5 major medical fields across 1604 Medicare-participating hospitals. A noticeable facet of the paper by Shahian et al. was their referencing of a large number of papers that we had identified either in our original review, or in the first phase of our updated review [ 40 ].

To ensure our updated review adequately reflected all such developments since May 2020, we conducted a further search in March 2024. The review presented here is based on papers identified in both phases of the updated review, the findings of which are then combined with those from our original review.

Review question

To identify studies, the primary research question used the same approach as Boaz et al. [ 2 , 3 ].

Does research engagement (by clinicians and organizations) improve healthcare performance?

By research engagement, we mean, as in our original review, engagement in research rather than the broader concept of engagement with research, and we are referring to participation in research by healthcare organizations and staff rather than patient participation in trials. Engagement in research is taken to mean, “a deliberate set of intellectual and practical activities undertaken by healthcare staff (including conducting research and playing an active role in the whole research cycle) and organizations (including playing an active role in research networks, partnerships or collaborations)” (p. 2) [ 2 ].

The 2020 decision to complete an update of the previous review [ 2 , 3 ] was informed by a published decision framework for updating systematic reviews [ 41 ]. After completion in 2024 of the comprehensive initial phase of the updated review, including the two searches and considerable subsequent analysis, we recognized, as noted above, that while we had been conducting the review some important further papers had been published. We wanted to incorporate such papers, and so decided to conduct a further search for papers. The design of this final phase (which included a third search) was informed both by the fact that we had already identified a considerable number of papers for the updated review, and by the way new papers in this field were by now much more likely to cite earlier papers, with Shahian et al. [ 40 ] being a prime example. Therefore, we thought it was reasonable to rely to a much greater extent on checking citations to the papers already identified, as explained below.

Search strategy and information sources

Search 1 (update).

The first step in syntax development used the Medline Ovid strategy published by Boaz et al. [ 2 ].

Initial diagnostic testing indicated issues preventing code execution. Due to the syntax comprising several nested terms and Boolean operators, it was rebuilt using recommendations for “single-line” optimization for debugging complex code [ 42 ].

Search 2 (modified)

The syntax for Search 2 was a term modification to capture papers that more explicitly indexed research networks and collaborations. Search 2 necessitated a deeper dive into the full-text content of papers. The decision to search full-text articles reflected observations that the sensitivity of Search 1 was potentially affected by the variable quality (and relevance for our review) of abstracts, a consistent challenge for reviewers [ 43 ]. As a second search also adapted published syntax, the Preferred Reporting Items for Systematic reviews and Meta-Analyses Literature Search—Extension Checklist (PRISMA-S) reporting protocol was followed [ 44 ]. (The full text for search strategies is provided in “Search Strategy and Syntax Sensitivity”; see Additional file 2 ).

Electronic databases

Nine electronic records collections were used in Search 1: Medline (OVID and EBSCO), EMBASE, PsycInfo (OVID and EBSCO), CINAHL, Web of Science, Health Management and Information Consortium and British Nursing Institute. The mix provided parity with previous reviews and mitigated risk of missed papers by combining general and specialized databases. Different interfaces (e.g. OVID, EBSCO) for the same collection were also included to offset variations due to platform [ 45 ]. Grey literature was not searched: these collections failed to uniquely identify papers in previous reviews on this topic. Search 2 was restricted to the Medline EBSCO Full Text records, which was the collection which yielded the highest hit ratio for relevant papers (see Additional file 2 ).

Other sources

Manual and snowball searching were used in three ways. Firstly, a range of search engines (Google Scholar, PubMed, ProQuest Central, Scopus, the Web of Science Cited Reference Search) were used to track citations for (a) prior reviews as whole papers, (b) the individual studies within these reviews and (c) article reference lists. Secondly, key journals that published studies shortlisted in the previous reviews were hand-checked, including: Implementation Science, PLOS One, BMJ Open and BMC Health Services Research. Thirdly, topic experts suggested papers for consideration.

Search 3 (final phase)

As explained above, we subsequently conducted a further search covering May 2020–March 2024. This consisted of: a hand-search of three of the journals in which papers from the first phase of the updated review had been published (Health Research Policy and Systems, Implementation Science and Medical Care); a check of papers in the two reviews published in this period [ 38 , 39 ]; and a check of citations in this period to all the papers identified both in our original review and in the update’s initial phase.

Eligibility criteria

The following limiters were applied:

Timeframe: 1 January 2012 to 20 March 2024 (inclusive of eprint)

Population: Human (any setting)

Language: English (any country)

Paper type: Academic Journals (scholarly works). Conference papers were admitted as flags for accessible peer-reviewed works (e.g. pre-print) or key teams.

Three criteria were defined, guided by definitions from the original review [ 3 ].

Criterion A: study design

Empirical studies using method/s aligned with health services research, including clinical trials, retrospective cohort and survey methods. Studies with only patient reported outcomes (e.g. satisfaction) were excluded.

Criterion B: healthcare performance

Studies must report an outcome indexing performance assessment for a care process or healthcare improvement. The following were excluded: staff-specific reports alone, (e.g. job satisfaction or morale), policy impacts alone (no flow through to healthcare), descriptions of networks without outcomes data.

Criterion C: research engagement

Explicit demonstration of engagement in research including: agenda-setting, conducting research, participation in action research or in networks where the research involvement is noted. This criterion also allowed engagement implicitly through research network membership, even if a specific study was not recorded, but there was a comparison of healthcare between member and non-member settings. More details about examples that were in scope can be found in Hanney et al. [ 3 , p. 2].

Records management

To efficiently manage the export of the large records for the first two searches, Endnote X9 (Clarivate) was used to combine downloads from different databases and discard software detected duplicates. The endnote library was imported into Rayyan, a free multi-collaborator online screening tool [ 46 ]. Study selection procedures for Searches 1 and 2 followed the same screening/eligibility check sequence.

Screening and eligibility/quality checking

In Rayyan, titles were scanned to exclude papers that were irrelevant, did not meet criteria or were non-exact duplicates. Abstracts of retained records were then screened and classified as “include”, “exclude” or “maybe”. A third screening of “maybe” classifications forced a binary coding of “include” or “exclude”, with comment flags on issues. A final records’ sweep with the Rayyan query function checked for misclassified studies. This four-step screening process was completed by a single reviewer (BG).

Full-text for each provisionally included study was uploaded into Rayyan. The initial eligibility check was completed by three experts who were involved in article screening for Hanney et al. [ 3 ]. As a criterion check and to orient reviewers to the Rayyan platform, a practice phase used 10 randomly sampled records. The abstract was the primary source for expert reviewers, with full-text also available. After the practice task and consensus discussion of criteria, a batch of records (alphabet determined) was assigned to each expert reviewer, to rate each paper as “include”, “exclude” or “maybe” (ratings were unblinded). If the rating pair (i.e. B.G. and an expert from the original review) were both “include”, the paper was progressed to full-text appraisal. If there was disagreement, papers rated as “maybe” were reassigned to another expert reviewer for an opinion, and those rated as “exclude” by an expert reviewer were marked for discard. If consensus for a “maybe” paper could not be reached by discussion, it was progressed to a full text appraisal, conducted by a single reviewer (B.G.) using all available information sources and reviewer ratings.

A final review of all potential “includes” was jointly conducted by team members, including a few papers identified by other sources such as continued manual snowballing from key papers. The study selection procedures for Search 3 mirrored this final step, and so consisted of a review of all potential “includes” conducted jointly by team members.

Study quality was assessed using the mixed-methods appraisal tool (MMAT v2018), on a scale of 1 (low) to 5 (high) [ 47 ]. The MMAT accommodated all designs in the paper set. The majority of the papers have a design which fitted into the MMAT category of quantitative non-randomized. All papers scored good to high quality on the five questions in their relevant MMAT subscale. The lower end of ratings (good) was typically due to lack of information in the article, such as whether and/or how confounding factors may have been identified or managed. Quality ratings were not used to exclude papers, but formed part of the discussion about the quality and contribution of the papers.

Data extraction, coding and ethics

As Rayyan is only a screening platform, a data extraction sheet was created in Excel (v2016) for each included paper. A university research ethics committee deemed the project as not requiring formal ethical approval, due to secondary data mining on anonymized aggregated records.

A large and methodologically diverse mix of papers was identified with a range of different outcomes and outcome measures. The papers were combined through a process of critical interpretive synthesis inspired, as in our original review, by the approach outlined by Dixon-Woods et al. [ 48 ]. This involves adopting an iterative approach to refining the research question, searching the literature and defining and applying codes and categories. It enables the generation and development of theory with strong explanatory power and uses relevance as one measure of quality. Following analysis of the papers in the updated review, we collated the results from the updated review with those from our original review to create one combined set of papers for overall analysis.

Figure  1 summarizes the review literature flow. The two formal searches identified 66 901 records, with 68 further papers coming from other sources, including the March 2024 extension. From these, 168 papers progressed to full-text appraisal, and 62 were identified for inclusion [ 40 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 ].

figure 1

Flow diagram for literature search

This review updates the previous review conducted by the team [ 2 ]. Table 2 outlines the 95 papers in our combined review: the 62 additional papers in the updated review along with the 33 papers in our original review. The latter 33 papers are shown in italics in Table  2 , which includes details about the study characteristics of all 95 included papers as well as key dimensions of the findings. To complement Table  2 , brief notes on the development and scope of key United States and United Kingdom research networks/organizations discussed in the papers are provided in the “Glossary of the United States of America and United Kingdom Research Organizations and Networks Discussed in the Papers” (see Additional file 1 ).

Study characteristics

Across the 95 papers, 12 countries are either the location for the research engagement described in a single-country study, or the location from which a multi-country study was led, with one paper led from South Africa having authors from a range of African countries (and Yemen) [ 88 ]. The 12 countries are: United States (50 papers), United Kingdom (17), Canada (7), Spain (5), Germany (4), the Netherlands (3), Australia (2), Denmark (2), South Africa (2), China (1), Finland (1) and Sweden (1).

Cancer was the most common field, with 32/95 papers overall. Next came hospital care in general/multi-field/acute care with 16 papers, cardiovascular/stroke (12), substance use disorder (7), dentistry (3), mental health/psychiatry (3) and obstetrics (3).

Main findings

As presented in Table  2 , the key findings from the combined review are presented in terms of the four pairs of binary options, though inevitably some papers did not neatly fit into one category. The first categorization is in terms of the level of analysis explored in different papers; 23 papers compare clinicians, but 72 compare organizations. There is an even higher proportion in the updated review at the organizational level (50/62, 81%) than in our original review (22/33, 67%).

A total of 86 of the 95 papers report positive results, of which 70 are purely positive and 16 are positive/mixed meaning that there are some key negative elements in their findings, that is, important parts of the analysis where a lack of healthcare improvement is identified. Nine papers are negative, of which four are negative-mixed.

The final two pairs of binary options consider just the 86 positive papers. In total, 37/86 report improved health outcomes in terms of reduced mortality or morbidity. A higher proportion of the positive papers in the updated review (30/58, 52%) than in our original review (7/28, 25%) describe such improved health outcomes. There is a corresponding reduction from three quarters (21/28) to a half (28/58) in the proportion of papers solely describing improved processes in terms such as applying proven interventions.

Finally, in terms of the type of impact, 55/86 of the papers describe research engagement leading to a broader impact on healthcare performance. Broad impacts arise when the improved healthcare goes more widely than just being linked to clinicians or healthcare organizations implementing the findings, or processes, from their own research more rapidly/extensively than do others. When the improved healthcare is linked to the results or processes of their own research, that is categorized as specific impact, which is the case in 31/86 papers. Using these various categories, Fig.  2 outlines the findings from the combined review, alongside the findings from our original review, and the updated review. This highlights various trends in terms of the main findings.

figure 2

Results from Boaz et al. systematic reviews of whether research engagement by health organizations and staff improves healthcare: analysis of original; updated; and combined reviews (and of the 86 positive papers). Green rows (top): original review; Brown rows (middle): updated review; blue rows (bottom): combined review

One further trend in terms of the type of analysis is seen in the 11/95 papers that used bibliometric analysis as an indicator of the extent, and/or quality, of research engagement compared with some measure of the healthcare performance, in terms of processes and/or outcomes [ 30 , 40 , 50 , 51 , 56 , 65 , 76 , 95 , 96 , 97 , 100 ]; 10 of these papers are in the updated review, with just 1 [ 30 ] from our original review. All these 11 papers are positive, but various types of bibliometric analysis are used. The broad categories of academic indicators applied include publication volume [ 95 , 96 ], publication “quality” (for example, as measured by citations) [ 51 ] and a combination of volume and “quality” [ 30 , 40 , 50 , 56 , 65 , 76 , 97 , 100 ]. Of the latter, five relatively small studies suggest that the association with “quality” was stronger than with volume. The bibliometric studies also illustrate the varying levels of analysis at which the included studies in the review are conducted; 4 of the 11 papers compare the academic outputs of clinicians [ 50 , 56 , 96 , 97 ] and 7 make comparisons at an organizational level [ 30 , 40 , 51 , 65 , 76 , 95 , 100 ], focussing variously on academic outputs at ward, department or hospital/trust level.

The combined review allows for a range of issues to be analysed more thoroughly than they had been in our original review. These include issues highlighted in the background such as the role of networks and the “dose effect”. These are examined in turn below, followed by consideration of how far the included studies have addressed various aspects of health equity, and finally an analysis of lessons from the overall portfolio of positive and negative studies.

The role of research networks

The full significance of papers on research networks is seen in the combined review. Using the inclusive definition developed by Laliberte et al. [ 24 ], we have applied the term to various arrangements that, however loosely, give some measure of commonality to the research of multiple healthcare organizations that not only enhance science production, but also share a concern to transfer research findings into clinical practice. About half the papers in the combined review analysed research activity by clinicians or healthcare organizations who were part of research networks of various types.

In the United States, the NCI cancer research networks include the NCI-designated Comprehensive Cancer Centres, the NCI Cooperative Groups and collaborative groups of community hospitals affiliated to the NCI’s CCOP- see the Glossary for its new name. In various ways these networks all include outreach and the engagement of community physicians in their brief; see the Glossary for more details. Their potential was recognized early in the 2005 study by Laliberte et al. [ 24 ] that looked at these networks and concluded that network membership may influence compliance with treatment guidelines, and should therefore be taken into account in predictive models of compliance.

Seven included papers illustrated various aspects of this issue by comparing the processes and outcomes for patients treated at NCI-designated (comprehensive) cancer centres with those treated elsewhere, six of these studies showed better outcomes for patients treated at NCI centres [ 52 , 64 , 80 , 84 , 86 , 106 ], while one paper suggested that despite better processes, patient outcomes were worse at NCI centres. This paper is considered in the section on negative papers below [ 81 ]. Of the positive papers, Paulson et al. showed how the NCI designation was “associated with lower risk of postoperative death and improved long-term survival” (p. 675) [ 86 ], identified possible factors such as better adherence to guidelines, and demonstrated that the better outcomes at NCI-designated centres remained even when compared with non-NCI designated centres with a similar high volume of cases [ 86 ]. Wolfson et al. identified the requirements that underpin the positive association between high-quality research and high-quality care [ 106 ]. These included the mandate NCI centres have to “lead clinical trials, exchange ideas, disseminate findings” (p. 3892), which showed how the centres could act as part of a network. Wolfson et al. continued: “The NCI operates on the belief that a culture of discovery, scientific excellence, transdisciplinary research, and collaboration yields tangible benefits extending far beyond the generation of new knowledge” [ 106 ].

Building on Laliberte et al. [ 24 ], Carpenter et al. demonstrated an association between CCOP membership and accelerated innovation adoption but added the important codicil that it was not possible to “definitively ascertain whether there is a direct causal relationship between the two” [ 54 ].

Improved healthcare has also been associated with membership of the United States practice-based research networks (PBRNs). These networks cover family practice/primary care, dentistry, mental health and substance abuse. Like the CCOP and its affiliates, PBRNs involve practising clinicians in the community who conduct research. The combined review includes seven PBRN papers covering primary care and dentistry, all of which are positive [ 32 , 36 , 66 , 78 , 83 , 92 , 108 ] and one of which describes an international dental PBRN led from the United States that includes three Scandinavian countries [ 66 ].

A total of seven papers from another PBRN, the National Institute on Drug Abuse’s Clinical Trials Network (CTN), also provided evidence of accelerated translation, identified mechanisms through which this might work, and discussed the theoretical frameworks within which those operated [ 5 , 14 , 23 , 49 , 63 , 90 , 91 ]. Thus, Ducharme et al. [ 14 ] and Knudsen et al. [ 23 ] explored Rogers’ notion of the “trialability” [ 110 ] of innovations, that is, how far an innovation may be experimented with on just a limited basis, and Abraham et al. [ 5 ] discussed the role of absorptive capacity [ 111 , 112 ], which they summarized as an organization’s ability to assess and use information [ 5 ]. Rieckmann et al. noted that although the mechanisms involved were not fully understood they appeared “to be influenced by core experiences from network participation” (p. 894) [ 91 ], and Fields et al. [ 63 ] used insights from implementation science to explore the influence of a set of organizational characteristics (including network membership) on innovation adoption [ 113 ].

In an analysis of data on 12 993 transplants conducted in 162 US centres, the 32 centres in the Bone Marrow Transplant trials network were found to have significantly better survival rates than others [ 77 ]. Marmor et al. reported that there was not an association between procedure volume and survival. Rather, they suggested, the better outcomes for those treated in centres in this network could be linked to the nature of trials that required “higher levels of national clinical collaboration and standardization of protocols”, and such collaboration was “likely to generate higher levels of innovation and excellence among clinical colleagues” (p. 92) [ 77 ].

In Germany, one team produced three papers on the improved healthcare performance of hospitals that were part of clinical trials organizations [ 13 , 34 , 94 ]. Two papers described the improved outcomes for patients with ovarian cancer if they were treated in a hospital that belonged to one of two German ovarian cancer clinical trials organizations, in effect research networks [ 13 , 34 ]. They noted that the improved outcomes were not related to patient volume, suggesting instead that possible factors may include hospitals’ participation in the study group’s quality assurance programs and team members attending regular and scientific and educational meetings [ 13 ]. In a follow-up study, the data were analysed in more detail using mediation analysis that showed not just that the research participation of a hospital contributed to superior patient survival, but also began to unpick how it happened, including through better use of surgery and chemotherapy [ 94 ].

Downing et al. noted that, following the 2006 establishment of the NIHR in the United Kingdom, the increase in research activity in networks throughout the English NHS also increased the scope for analysing the benefits of research engagement [ 58 ]. The role of NIHR networks in boosting research engagement, which is then linked to improved healthcare, also covers clinicians such as nurses and AHPs who had traditionally had limited research opportunities. Studies are now showing how they can play an important role by engaging in research because, according to Trusson et al. reporting on a research network for nurses and AHPs, people working in such roles “have opportunities to explore possible solutions to issues that they encounter in their clinical role through academic study” (p. 1) [ 101 ]. Such opportunities can also enhance their clinical skills. More broadly, Downing et al. claimed that, in relation to the NIHR’s clinical trials network, “this natural experiment, presented by the rapid expansion of trial activity across a whole national health system, is perhaps the best opportunity to address the subject though outcomes research” (p. 95) [ 58 ]. This development is discussed in the next section.

The “dose effect” of the extent of research engagement

Evidence indicating a link between the extent of research engagement and the degree of improved healthcare has been accumulating for some time. In the United States, the 1996 study by Brown and Griffiss found that the average acute length of stay (LoS) in Department of Veteran Affairs hospitals was inversely related to the size of research programmes [ 53 ]. Majumdar et al. [ 26 ] used a tertile approach to show that in-hospital mortality decreased as the rate of trial participation increased in the area of unstable angina. In the substance abuse field, early CTN studies also contributed: thus Knudsen et al. [ 23 ] noted that the adoption of buprenorphine therapy by practitioners within the trials’ network was much greater in those programmes in the network that participated in the specific buprenorphine trial than those that had not. In a 2006 study of a sexual health trial in Australia, Morton et al. [ 28 ] identified improved post-trial clinical practice by high-recruiting clinicians, but not by low-recruiting ones.

In our combined set of papers the first use of the specific term “dose effect” to describe the effects of differing amounts of research engagement occurred in Downing et al., who tested the hypothesis that for colorectal cancer (CRC) “high, sustained hospital-level participation in interventional clinical trials improves outcomes for all patients with CRC managed in those research-intensive hospitals” (p. 89) [ 58 ]. They found that high participation in such clinical trials was independently associated with better outcomes and that these effects were not restricted to academic centres or large institutions but were seen across all the NHS Trusts that conducted research on and treated patients with colorectal cancer. They extended their analysis to look at the effects of different levels of research participation and found that the highest levels of participation led to the highest levels of improved outcomes. However, in relation to these findings, Downing et al. were careful to say that, in the absence of the possibility of an RCT, caution was needed if attempting “to infer a causal contribution” (p. 89) from participation in research activity to improved healthcare [ 58 ].

Other United Kingdom database studies support the findings of Downing et al. For example, Ozdemir et al. [ 85 ] compared mortality with research funding per hospital bed in hospitals with high, medium and low levels of research funding and showed that not only was mortality lower in high-funded research hospitals than in other hospitals, but also, on average, hospitals in the middle category had a lower mortality rate than ones with the least research funding. In two studies using NIHR research study activity data from different years, Jonker and Fisher [ 68 , 69 ] showed an inverse correlation between the number of clinical trials/patient participation levels in United Kingdom hospitals and the mortality rate. Lin et al. [ 73 ] used retrospective data to examine the survival rate of the 465 patients (recruited by 60 hospitals) who had participated in an RCT in the NIHR Clinical Research Network (CRN). While they identified a significant association between low trial recruitment and lower survival rates, looking at the volume of patients treated in the disease area by the respective hospitals they report that “no significance was found between hospital throughput and outcomes” (p. 40) [ 73 ].

Further support for the “dose effect” concept comes from the United States and elsewhere. According to Abraham et al., in the substance abuse field “treatment programs participating in a greater number of CTN protocols had significantly higher levels of treatment quality, an association that held after controlling for key organizational characteristics” (p. 232) [ 49 ]. Similarly, Gilbert et al. [ 66 ] reported that members of a dental PBRN who fully participated in the network were more likely to move evidence-based care into everyday practice than members who only partially participated. Seaburg et al. [ 96 ] showed an association between the quantity of resident physicians’ publications and their clinical performance scores during training, and García-Romero et al. claimed that increases in the scientific output of Spanish hospitals made a significant contribution to a reduction of hospital LoS [ 65 ].

In Canada, Tsang et al. [ 103 ] conducted a pre-planned observational study nested within a clinical trial to test how well traditionally non-research active community hospitals could participate in an RCT alongside the traditional RCT sites in academic hospitals. However, while that aspect of the study did show that, in terms of adherence to trial metrics, the community hospitals could successfully participate in studies, outcomes for patients in the trial were significantly better in the traditional research hospitals, although the full reasons for this will need further exploration [ 103 ].

Various aspects of health equity are considered in the included papers, and some of these report attempts to improve health equity. Some population groups are particularly vulnerable. In the United States, for example, Wolfson et al. listed the following groups: “underrepresented minorities, those with low socio-economic status (SES), those with public or no insurance, and those with a significant distance to care” (p. 3886) [ 106 ]. On the basis of its long-held assumption that patient access to research active healthcare providers is beneficial, the NCI has attempted to reduce geographic inequalities in access. In a 1995 paper, Warneke et al. noted that the CCOP was established by the NCI in 1983 with the deliberate intention of spreading the benefits of the clinical research conducted in NCI centres: “The program was designed with the assumption that by participating as equals in the research process, community physicians would be more likely to accept and implement the results in their practices with non-protocol patients” (p. 336) [ 37 ].

Similar moves to encourage wider participation in clinical trials have recently been made in Canada in the nested study described above [ 103 ]. A recent analysis showing higher levels of research activity within the English healthcare system were associated with lower mortality, noted that although the NIHR CRN was established to promote research participation across England, there was still some way to go to ensure greater geographical equity [ 69 ].

Other initiatives, such as the United States minority-based CCOPs described in the Glossary, addressed racial inequalities in relation to access to research engagement and timely evidence-based healthcare. These sometimes overlap with geographic inequalities. Some of the papers on the NCI-designated cancer centres observed with concern that the proportion of certain racial/ethnic groups, including African Americans, who received treatment at these centres compared with non-NCI centres, was lower than for other racial groups [ 64 , 80 , 106 ]. Having noted that African Americans with colon cancer experienced worse outcomes than Caucasian Americans, and suggested that this was partly due to differential treatment, a study by Penn et al. found evidence that African Americans receiving treatment from CCOP providers had benefitted from a seemingly deliberate attempt to boost early access to a recently recommended innovative treatment [ 87 ]. In Australia, Young et al. [ 109 ] reported that the health services, and health research system, of the Aboriginal community work together to try to ensure health research is embedded into activities that improve health, and described a specific example in relation to ear, nose and throat surgery and speech-language pathology services.

Lessons from the overall collection of studies: positive and negative

A wide variety of papers contribute to the combined review’s overall finding that the included studies are overwhelmingly positive. As the section on the “dose effect” illustrates, throughout the time covered by the combined review, individual papers have contributed to a wider understanding that goes beyond specific issues about research networks. Many papers contribute to the analysis of both the strength of the association between research engagement and improved healthcare, and the mechanisms involved. For example, a 2019 US positive study by Fanaroff et al. [ 60 ] identified improved care and outcomes for patients with acute myocardial infarction who were treated at research active hospitals, even after accounting for potential confounders. The authors encapsulated some of the key thinking on research engagement with their conclusion that participation in clinical trials by hospitals “may be emblematic of a culture that embraces novel therapeutics, engages both clinicians and patients, and incentivizes continuous improvement in care” (p. 191) [ 60 ].

While overall the 95 studies included in the combined review are positive, about 10% are categorized as negative. These nine negative papers also provide important insights [ 7 , 11 , 15 , 20 , 25 , 67 , 79 , 81 , 99 ]. For example, existing widespread use of one proven intervention prior to a company-sponsored clinical trial exploring physicians’ adherence to international treatment recommendations meant that the trial had no significant impact on that adherence, although it did increase use of the trial sponsor’s drug [ 7 ]; physicians adopted another trial intervention before it was proven one way or another [ 11 ]; more positively, a unique policy and regulatory environment governing the adoption of another intervention ensured that all hospitals benefitted, not just those in the trial [ 79 ]. Two teams with negative results later conducted further, more comprehensive studies with positive conclusions [ 25 , 26 , 67 , 68 , 69 ]. Six of the seven papers examining whether NCI-designated cancer centres provided patients with better healthcare processes and outcomes are positive [ 52 , 64 , 80 , 84 , 86 , 106 ]. However, one paper suggested that outcomes were worse in these accredited hospitals despite the better healthcare and, in seeking to explain this, drew attention to the factors considered in the accreditation processes used by different organizations and how far they accurately captured the most relevant data [ 81 ].

Our original review set out to find whether there was empirical evidence that supported the often-held assumption that engagement by clinicians and healthcare organizations in research improves healthcare performance at various levels. It concluded that there was some positive evidence but that systematic analysis of the data related to this engagement was in its infancy [ 2 ]. The 62 papers in the updated review, 58 of which are positive, provide further empirical evidence to support the positive conclusions of the original review.

When the papers from both reviews are considered together, they provide a more complete dataset than previously available [ 1 , 2 , 3 , 4 ], and an updated picture of this literature in which the trends identified in our initial analyses [ 3 ] become more apparent. With more than a third of the papers in the combined review (32/95) focussing on aspects of cancer, this is the field overall in which there is the most comprehensive analysis of the link between research engagement and improved healthcare. While the individual cancer papers differ in the strength of the association identified, and most of the papers focus one or other of the main cancer sites, many of the cancer papers analyse the role of research networks – one of the main mechanisms through which it is claimed research engagement improves healthcare.

The combined review reflects policy shifts and organizational changes that occurred first in the United States and later in the United Kingdom and elsewhere, and were designed to address the time lag between the production of research and its use in practice. These include the development of research networks and their associated databases over several decades (accompanied by an improved understanding of their strengths and limitations [ 54 , 64 , 77 , 106 , 108 ]) and efforts to strengthen links between academic centres and community services [ 61 , 87 ]. More recent developments, especially in the United Kingdom, encouraged further deliberate attempts to identify and explore the impacts of research engagement. Research teams were, for example, better able to study the real-world impacts of system-level mechanisms such as research networks as they became more formalized and embedded in national health and science structures [ 58 , 67 , 68 , 69 , 85 , 93 , 101 ].

Across the board, within and beyond networks, there is also further evidence about the mechanisms by which research engagement might improve healthcare, including the ones identified in our original review. The role of strong evidence-based protocols developed for RCTs, but contributing to improved healthcare more widely in research active healthcare sites, was highlighted in various studies [ 77 , 98 , 105 ]. Papers also identified the importance of providing evidence-based/guideline consistent care, which could also be linked to a culture of discovery, excellence and collaboration [ 40 , 60 , 62 , 64 , 77 , 84 , 86 , 87 , 106 ]. There were also more nuanced mechanisms at the speciality and clinician levels, such as the use of multi-disciplinary coordination of care in radiation therapy treatment [ 107 ] and practitioner skill development in substance abuse work [ 90 ]. Similar practitioner skill development was also reported among nurses and AHPs, including in the wider literature [ 31 , 38 , 39 , 62 , 70 , 101 ].

In the combined review it also became easier to see connections across this diverse literature. It was possible to identify research teams that had worked together on multiple studies and to explore the extent of cross referencing. In the United States, for example, the CTN of the drug abuse institute had been created to emulate the CCOP, and a centre was established to assess the CTN’s impact [ 114 ]. Analysis of this research network highlighted its role both in conducting research that was relevant to the “real-world” needs of clinical settings, and in enhancing evidence-adoption by healthcare organizations and staff [ 114 ]. Many of the papers from this substance abuse CTN [ 23 , 49 , 90 ] referenced each other and also cross-referenced key cancer papers [ 8 , 24 , 54 ], and there was common use of the same early sources [ 110 , 112 , 115 , 116 ]. These interactions prompted ongoing methodological development, strengthened understanding of theoretical concepts, and supported shared learning across the specialities. Additionally, themes that had been recognized in the original review, including concepts such as absorptive capacity [ 5 , 111 ], were further explored and tested in new contexts, even if the same literature was not always drawn upon [ 40 , 65 ].

In the combined review, the nature and strength of the association found between research engagement and improved health varies enormously among the 86 positive papers, even among those that describe the role of research networks. One approach that begins to identify where evidence might be strongest was noted in the original review as being the important concept of the “dose effect”, even if it was not specifically labelled as such [ 26 ]. However, the combined review can now more fully consider the concept because evidence about this greatly increased as the scope of the papers included has increased. There are many more studies where all the clinicians or organizations compared are engaged in research but to varying extents and/or with different levels of resources, for example within a trial [ 28 , 50 , 59 , 73 , 93 , 103 , 107 ] or within a network [ 23 , 33 , 49 , 51 , 66 , 68 , 69 , 85 , 95 ]. The inclusion of papers regarding differences within trials, and the emergence of the importance of the “dose effect”, have implications for both (a) how the issue of research engagement is analysed and (b) how far efforts to enhance research engagement should be concentrated or spread widely across a system.

In relation to the first of these issues, when considering how research engagement is analysed, the key question morphs somewhat: it is no longer simply whether research engagement improves healthcare performance compared with no research engagement, rather, it is whether a larger amount of research engagement improves healthcare performance by more than a smaller level of engagement (and, if so, by how much). Answers to these questions could then feed back to strengthen the evidence for a positive association between research engagement and improved healthcare performance.

In relation to the second question, about the concentration or wide distribution of research funding, analyses might have to consider the context and trade-offs in terms of benefits for improved health and health equity. The widespread distribution of research funding across the health system could maximize the number of patients who might benefit, but a more concentrated approach, with a higher dose of research engagement in a smaller number of hospitals, could maximize the benefit for patients in such centres.

Research infrastructures in countries such as the United States and United Kingdom have been developed to enhance the relationship between health and health research systems, and the evidence from our combined review suggests that these changes have been positive. In both systems, but particularly in the United Kingdom, there have been deliberate attempts to fund major centres of research in leading healthcare facilities, as well as to spread research funding more widely to healthcare organizations across the country, but this impetus needs to be maintained if the full benefits of research engagement are to be realized.

Such an argument is reinforced by the conclusions of a major recent analysis of progress in the United Kingdom in engaging healthcare staff in research and building research capacity. The findings from the study suggest that many healthcare staff in the United Kingdom are interested in being involved in research, there are supportive national policies and strategies in place and there has been some important progress. However, achieving widespread involvement “will only be possible by focusing more on how healthcare organizations embed and support research activity through organizational policies which are supported by the wider research support and funding infrastructure. This is an essential part of a system-based approach to developing and supporting research engagement” (p. 356) [ 117 ]. The progress possible, and the potential benefits of trying to build a health research system embedded into a healthcare system, but also the full range of substantial challenges, have also recently been explored in a hospital and regional healthcare system in northern Queensland, Australia [ 118 , 119 ]. Studies such as these indicate that this combined review could provide timely evidence to further the challenging task of improving healthcare by boosting engagement in health research.

Strengths and limitations

The combined review contains a considerable number of papers from diverse perspectives, but the literature is drawn predominantly from the United States and the Global North, thus the conclusions may not be appropriate in different contexts, including in the Global South. This, perhaps, partly reflects the inclusion criteria of papers in English only. While the increasing use of bibliometrics as an indicator of research engagement has widened the range of positive studies available, differing claims as to the most appropriate measure of research publications challenge consistent interpretation of the data and indicate there is more work to do. Furthermore, it is important to recognize that the national policy, noted in one paper, of attaching promotion and bonuses for clinicians to publish in journals with an impact factor of at least three [ 97 ] runs contrary to the internationally widely endorsed Declaration on Research Assessment [ 120 ].

The complexity of this literature (with many generic terms such as “research” and “engagement”), and the tangential approach of some papers to the broad question of whether research engagement improves performance, posed considerable challenges. It helped enormously that this time around, we were able to build on our experience in the original review. We adopted a somewhat more extensive approach to the formal search in the updated review, and we identified some papers that we had missed in the original review. We were aided by the generally greater clarity in later papers. We are now able, therefore, to present a more nuanced understanding of this field, building on our experience in the original review. In particular, we have found considerably more evidence on two topics identified as important in our original review, and on their implications for health equity: the role of research networks and consideration of how far there is a dose effect with regard to the degrees of research engagement. On both topics the combined review has strong papers showing important healthcare improvements even after considering potential confounders such as patient volume [ 8 , 13 , 26 , 40 , 58 , 73 , 77 , 84 , 85 , 86 ]. However, the failure of some papers to address such confounders [ 59 , 107 ] means some weaknesses in the overall analysis remain, and we are still not able to undertake any meta-analysis as the included literature remains very diverse.

We have now included a significant range of largely positive papers in the combined review. However, lack of resources meant we were not able to replicate our original review’s [ 2 , 3 ] structured analysis of the wider range of papers identified as making many relevant and illuminating points related to the topic, but not meeting the review’s inclusion criteria. For example, while the combined review does include some consideration of health equity issues, there were papers taken to full paper review that were not in the end included but which provide considerably more evidence [ 121 , 122 ].

Future possible work

The system-based approaches for expanding the amount of research in healthcare systems that are mentioned above continue to provide important opportunities for further work on exploring the relationship between research engagement and improved healthcare, including the implications for health equity. Likewise, improvements in the identification and collection of relevant data and developments in statistics have prompted increasingly sophisticated analyses, sometimes using approaches developed in other fields, and could continue to do so [ 65 , 90 , 94 ]. There has also been increasingly sophisticated use of bibliometrics, and there are likely to be continuing opportunities to apply such approaches to more countries. However, the warning from Downing et al. that caution is needed if attempting “to infer a causal contribution” from research participation to improved health outcomes [ 58 ], as well as frequent mention of similar disclaimers in other papers [ 8 , 40 , 54 , 59 , 69 , 74 ], is a reminder that more work is needed.

While some of our papers have claimed that the costs of research engagement are broadly covered by the associated reduced LoS [ 53 , 65 ], further research might be useful around the costs associated with research engagement and how these relate to reported benefits. Such studies could add to the existing large-scale studies showing the considerable monetary value of the health and economic gains resulting from health research [ 123 ].

The insights revealed by the negative papers, particularly in relation to the contexts in which research and research networks operate [ 99 ], could usefully be further explored. Merkow et al. [ 81 ], the one negative paper out of seven papers included on the NCI-designated centres, raises issues about the accuracy, or perhaps appropriateness, of the measurement used by various organizations to accredit cancer centres. These issues have also been explored by various teams [ 122 , 124 , 125 ] but could perhaps be worth further examination because the findings from Merkow et al. are so starkly different from those of other papers included in our review.

Finally, there are increasing opportunities, as well as a growing need, to address the limitations identified above (and also noted in the review by Chalmers et al. [ 39 ]) and go beyond the formal inclusion criteria of this review. A major area that could usefully be incorporated into an overall analysis of the field relates to the impact of the growing interest in research engagement strategies [ 126 ]. This includes the efforts to enhance research roles for healthcare professionals other than medical professionals [ 38 ], and the increasing number of organizational arrangements within health and health research systems for partnerships that seek to boost the production and use of relevant evidence [ 127 , 128 ].

Previous reviews [ 1 , 2 , 3 , 4 ] have investigated the association between research engagement and improvements in healthcare performance. This study updates and extends the most comprehensive of these reviews [ 2 , 3 ], and combines its findings with those from that original review to produce a more substantial pool of studies, which are largely positive in terms of the impact of research engagement on processes of care and patient outcomes. Of potential mechanisms, the combined review highlights the important role played by research networks and further identifies the various ways the research engagement facilitated by them operates to improve healthcare. The review also draws together a set of papers which consider how far there is a research engagement “dose effect”. Given the difficulty of conducting randomized controlled trials of large-scale research engagement initiatives, studies of the dose effect offer another approach to understanding the potential contribution and complexities of research engagement, including the implications for health equity. This review provides further evidence of the important contribution played by systems-level research investments such as research networks on processes of care and patient outcomes.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

Allied health professionals

Clinical research network

Clinical trials network

Community Clinical Oncology Program

Length of stay

Mixed-methods appraisal tool

National Cancer Institute

National Health Service

National Institute for Health (and Care) Research

Practice-based research network

Research and development

Randomized controlled trials

Clarke M, Loudon K. Effects on patients of their healthcare practitioner’s or institution’s participation in clinical trials: a systematic review. Trials. 2011;12:16. https://doi.org/10.1186/1745-6215-12-16 .

Article   PubMed   PubMed Central   Google Scholar  

Boaz A, Hanney S, Jones T, Soper B. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5: e009415. https://doi.org/10.1136/bmjopen-2015-009415 .

Hanney S, Boaz A, Jones T, Soper B. Engagement in research: an innovative three-stage review of the benefits for health-care performance. Health Serv Deliv Res. 2013;1:8. https://doi.org/10.3310/hsdr01080 .

Article   Google Scholar  

Harding K, Lynch L, Porter J, Taylor NF. Organisational benefits of a strong research culture in a health service: a systematic review. Aust Health Rev. 2017;41:45–53. https://doi.org/10.1071/AH15180 .

Article   PubMed   Google Scholar  

Abraham AJ, Knudsen HK, Rothrauff TC, Roman PM. The adoption of alcohol pharmacotherapies in the clinical trials network: the influence of research network participation. J Subst Abuse Treat. 2010;38:275–83. https://doi.org/10.1016/j.jsat.2010.01.003 .

Adler MW. Changes in local clinical practice following an experiment in medical care: evaluation of evaluation. J Epidemiol Community Health. 1978;32:143–6. https://doi.org/10.1136/jech.32.2.143 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Andersen M, Kragstrup J, Sondergaard J. How conducting a clinical trial affects physicians’ guideline adherence and drug preferences. JAMA. 2006;295:2759–64. https://doi.org/10.1001/jama.295.23.2759 .

Article   CAS   PubMed   Google Scholar  

Carpenter WR, Reeder-Hayes K, Bainbridge J, Meyer A-M, Amos KD, Weiner BJ, et al. The role of organizational affiliations and research networks in the diffusion of breast cancer treatment innovation. Med Care. 2011;49:172–9. https://doi.org/10.1097/MLR.0b013e3182028ff2 .

Chaney EF, Rubenstein LV, Liu C-F, Yano EM, Bolkan C, Lee M, et al. Implementing collaborative care for depression treatment in primary care: a cluster randomized evaluation of a quality improvement practice redesign. Implement Sci. 2011;6:121. https://doi.org/10.1186/1748-5908-6-121 .

Chen AY, Schrag N, Hao Y, Flanders WD, Kepner J, Stewart A, et al. Changes in treatment of advanced laryngeal cancer 1985–2001. Otolaryngol Head Neck Surg. 2006;135:831–7. https://doi.org/10.1016/j.otohns.2006.07.012 .

Clark WF, Garg AX, Blake PG, Rock GA, Heidenheim AP, Sackett DL. Effect of awareness of a randomized controlled trial on use of experimental therapy. JAMA. 2003;290:1351–5. https://doi.org/10.1001/jama.290.10.1351 .

Das D, Ishaq S, Harrison R, Kosuri K, Harper E, Decaestecker J, et al. Management of Barrett’s esophagus in the UK: overtreated and underbiopsied but improved by the introduction of a national randomised trial. Am J Gastroenterol. 2008;103:1079–89. https://doi.org/10.1111/j.1572-0241.2008.01790.x .

du Bois A, Rochon J, Lamparter C, Pfisterer J, for the Organkommission OVAR. Pattern of care and impact of participation in clinical studies on the outcome in ovarian cancer. Int J Gynecol Cancer. 2005;15:183–91. https://doi.org/10.1111/j.1525-1438.2005.15202.x .

Ducharme LJ, Knudsen HK, Roman PM, Johnson JA. Innovation adoption in substance abuse treatment: exposure, trialability, and the clinical trials network. J Subst Abuse Treat. 2007;32:321–9. https://doi.org/10.1016/j.jsat.2006.05.021 .

Goldberg HI, Neighbor WE, Hirsch IB, Cheadle AD, Ramsey SD, Gore E. Evidence-based management: using serial firm trials to improve diabetes care quality. Jt Comm J Qual Improv. 2002;28:155–66.

PubMed   Google Scholar  

Hall C, Sigford B, Sayer N. Practice changes associated with the Department of Veterans Affairs’ Family Care Collaborative. J Gen Int Med. 2010;25(Suppl. 1):18–26. https://doi.org/10.1007/s11606-009-1125-3 .

Hébert-Croteau N, Brisson J, Latreille J, Blanchette C, Deschenes L. Variations in the treatment of early-stage breast cancer in Quebec between 1988 and 1994. CMAJ. 1999;161:951–5.

PubMed   PubMed Central   Google Scholar  

Janni W, Kiechle M, Sommer H, Rack B, Gauger K, Heinrigs M, et al. Study participation improves treatment strategies and individual patient care in participating centers. Anticancer Res. 2006;26:3661–7. https://doi.org/10.1016/S0960-9776(05)80107-9 .

Jha P, Deboer D, Sykora K, Naylor CD. Characteristics and mortality outcomes of thrombolysis trial participants and nonparticipants: a population-based comparison. J Am Coll Cardiol. 1996;27:1335–42. https://doi.org/10.1016/0735-1097(96)00018-6 .

Jones B, Ratzer E, Clark J, Zeren F, Haun W. Does peer-reviewed publication change the habits of surgeons? Am J Surg. 2000;180:566–9. https://doi.org/10.1016/S0002-9610(00)00495-5 .

Karjalainen S, Palva I. Do treatment protocols improve end results? A study of survival of patients with multiple myeloma in Finland. BMJ. 1989;299:1069–72. https://doi.org/10.1136/bmj.299.6707.1069 .

Kizer JR, Cannon CP, McCabe CH, Mueller HS, Schweiger MJ, Davis VG, et al. Trends in the use of pharmacotherapies for acute myocardial infarction among physicians who design and/or implement randomized trials vs physicians in routine clinical practice: the MILIS-TIMI experience. Am Heart J. 1999;137:79–92. https://doi.org/10.1016/s0002-8703(99)70462-x .

Knudsen HK, Abraham AJ, Johnson JA, Roman PM. Buprenorphine adoption in the national drug abuse treatment clinical trials network. J Subst Abuse Treat. 2009;37:307–12. https://doi.org/10.1016/j.jsat.2008.12.004 .

Laliberte L, Fennell ML, Papandonatos G. The relationship of membership in research networks to compliance with treatment guidelines for early-stage breast cancer. Med Care. 2005;43:471–9. https://doi.org/10.1097/01.mlr.0000160416.66188.f5 .

Majumdar SR, Chang W-C, Armstrong PW. Do the investigative sites that take part in a positive clinical trial translate that evidence into practice? Am J Med. 2002;113:140–5. https://doi.org/10.1016/S0002-9343(02)01166-X .

Majumdar SR, Roe MT, Peterson ED, Chen AY, Gibler WB, Armstrong PW. Better outcomes for patients treated at hospitals that participate in clinical trials. Arch Intern Med. 2008;168:657–62. https://doi.org/10.1001/archinternmed.2007.124 .

Meineche-Schmidt V, Hvenegaard A, Juhl HH. Participation in a clinical trial influences the future management of patients with gastro-oesophageal reflux disease in general practice. Aliment Pharmacol Ther. 2006;24:1117–25. https://doi.org/10.1111/j.1365-2036.2006.03046.x .

Morton AN, Bradshaw CS, Fairley CK. Changes in the diagnosis and management of bacterial vaginosis following clinical research. Sex Health. 2006;3:183–5. https://doi.org/10.1071/SH06024 .

Pancorbo-Hidalgo PL, Garcia-Fernandez FP, Lopez-Medina IM, Lopez-Ortega J. Pressure ulcer care in Spain: nurses’ knowledge and clinical practice. J Adv Nurs. 2007;58:327–38. https://doi.org/10.1111/j.1365-2648.2007.04236.x .

Pons J, Sais C, Illa C, Méndez R, Suñen E, Casas M, et al. Is there an association between the quality of hospitals’ research and their quality of care? J Health Serv Res Policy. 2010;15:204–9. https://doi.org/10.1258/jhsrp.2010.009125 .

Puoane T, Sanders D, Ashworth A, Chopra M, Strasser S, McCoy D. Improving the hospital management of malnourished children by participatory research. Int J Qual Health Care. 2004;16:31–40. https://doi.org/10.1093/intqhc/mzh002 .

Rhyne R, Sussman AL, Fernald D, Weller N, Daniels E, Williams RL, et al. Reports of persistent change in the clinical encounter following research participation: a report from the primary care multiethnic network (PRIME Net). J Am Board Fam Med. 2011;24:496–502. https://doi.org/10.3122/jabfm.2011.05.100295 .

Rich AL, Tata LJ, Free CM, Stanley RA, Peake MD, Baldwin DR, et al. How do patient and hospital features influence outcomes in small-cell lung cancer in England? Br J Cancer. 2011;105:746–52. https://doi.org/10.1038/bjc.2011.310 .

Rochon J, du Bois A. Clinical research in epithelial ovarian cancer and patients’ outcome. Ann Oncol. 2011;22(Suppl. 7):vii16–9. https://doi.org/10.1093/annonc/mdr421 .

Salbach NM, Guilcher SJ, Jaglal SB, Davis DA. Determinants of research use in clinical decision making among physical therapists providing services post-stroke: a cross-sectional study. Implement Sci. 2010;5:77. https://doi.org/10.1186/1748-5908-5-77 .

Siegel RM, Bien J, Lichtenstein P, Davis J, Khoury JC, Knight JE, et al. A safety-net antibiotic prescription for otitis media: the effects of a PBRN study on patients and practitioners. Clin Pediatr. 2006;45:518–24. https://doi.org/10.1177/0009922806290567 .

Warnecke R, Johnson T, Kaluzny A, Ford L. The community clinical oncology program: its effect on clinical practice. Jt Comm J Qual Improv. 1995;21:336–9.

CAS   PubMed   Google Scholar  

Newington L, Wells M, Adonis A, Bolton L, Bolton Saghdaoui L, et al. A qualitative systematic review and thematic synthesis exploring the impacts of clinical academic activity by healthcare professionals outside medicine. BMC Health Serv Res. 2021;21:400. https://doi.org/10.1186/s12913-021-06354-y .

Chalmers S, Hill J, Connell L, Ackerley S, Kulkarni A, Roddam H. The value of allied health professional research engagement on healthcare performance: a systematic review. BMC Health Serv Res. 2023;23:766. https://doi.org/10.1186/s12913-023-09555-9 .

Shahian DM, McCloskey D, Liu X, Schneider E, Cheng D, Mort EA. The association of hospital research publications and clinical quality. Health Serv Res. 2022;57(3):587–97. https://doi.org/10.1111/1475-6773.13947 .

Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Bayene J, et al. When and how to update systematic reviews: consensus and checklist. BMJ. 2016;354: i3507. https://doi.org/10.1136/bmj.i3507 .

Bramer WM, De Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc. 2018;106:531–41. https://doi.org/10.1136/bmj.i3507 .

Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, et al. PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10:e1001419. https://doi.org/10.1371/journal.pmed.1001419 .

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Sys Rev. 2021;10:39. https://doi.org/10.1186/s13643-020-01542-z .

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Info Libr J. 2009;26:126–35. https://doi.org/10.1111/j.1471-1842.2008.00785.x .

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—A web and mobile app for systematic reviews. Syst Rev. 2016;5:210. https://doi.org/10.1186/s13643-016-0384-4 .

Hong QN, Pluye P, Fàbregues S, Bartlett G, et al. The mixed methods appraisal tool (MMAT) version 2018. User guide. http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf . Accessed 7 July 2024.

Dixon-Woods M, Cavers D, Agarwal S, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006;6:35. https://doi.org/10.1186/1471-2288-6-35 .

Abraham AJ, Knudsen HK, Roman PM. The relationship between clinical trial network protocol involvement and quality of substance use disorder treatment. J Subst Abuse Treat. 2014;46:232–7. https://doi.org/10.1016/j.jsat.2013.08.021 .

Alotaibi NM, Ibrahim GM, Wang J, Guha D, Mamdani M, Schweizer TA, et al. Neurosurgeon academic impact is associated with clinical outcomes after clipping of ruptured intracranial aneurysms. PLoS ONE. 2017;12: e0181521. https://doi.org/10.1371/journal.pone.0181521 .

Bennett WO, Bird JH, Burrows SA, Counter PR, Reddy VM. Does academic output correlate with better mortality rates in NHS trusts in England? Public Health. 2012;126(Suppl 1):S40–3. https://doi.org/10.1016/j.puhe.2012.05.021 .

Birkmeyer NJ, Goodney PP, Stukel TA, Hillner BE, Birkmeyer JD. Do cancer centers designated by the National Cancer Institute have better surgical outcomes? Cancer. 2005;103(3):435–41. https://doi.org/10.1002/cncr.20785 .

Brown A, Griffiss M. Effect of integrated research programs on health care systems and costs. Mil Med. 1996;161:691–5. https://doi.org/10.1093/milmed/161.11.691 .

Carpenter WR, Meyer AM, Wu Y, Qaqish B, Sanoff HK, Goldberg RM, et al. Translating research into practice: the role of provider-based research networks in the diffusion of an evidence-based colon cancer treatment innovation. Med Care. 2012;50:737–48. https://doi.org/10.1097/MLR.0b013e31824ebe13 .

Corrigan MH, Glass HE. Physician participation in clinical studies and subsequent prescribing of new drugs. Pharm Ther. 2005;30(1):60–6.

Google Scholar  

de Arriba-Enriquez J, Sanz-Casado E, Vieta E, Rapado-Castro M, Arango C. Quality of care in psychiatry is related to research activity. Eur Psychiatry. 2021;64(1): e53. https://doi.org/10.1192/j.eurpsy.2021.16 .

de Lange TS, Roos C, Bloemenkamp KW, Bolte AC, Duvekot JJ, Franssen MT, et al. Impact of a randomized trial on maintenance tocolysis on length of hospital admission of women with threatened preterm labor in the Netherlands. Eur J Obstet Gynecol Reprod Biol. 2015;186:8–11. https://doi.org/10.1016/j.ejogrb.2014.12.003 .

Downing A, Morris EJ, Corrigan N, Sebag-Montefiore D, Finan PJ, Thomas JD, et al. High hospital research participation and improved colorectal cancer survival outcomes: a population-based study. Gut. 2017;66:89–96. https://doi.org/10.1136/gutjnl-2015-311308 .

Eaton BR, Pugh SL, Bradley JD, Masters G, Kavadi VS, Narayan S, et al. Institutional enrollment and survival among NSCLC patients receiving chemoradiation: NRG oncology radiation therapy oncology group (RTOG) 0617. J Natl Cancer Inst. 2016;108(9): djw034. https://doi.org/10.1093/jnci/djw034 .

Fanaroff AC, Vora AN, Chen AY, Mathews R, Udell JA, Roe MT, et al. Hospital participation in clinical trials for patients with acute myocardial infarction: results from the National Cardiovascular Data Registry. Am Heart J. 2019;214:184–93. https://doi.org/10.1016/j.ahj.2019.05.011 .

Farquhar DR, Masood MM, Lenze NR, Sheth S, Patel SN, Lumley C, et al. Academic affiliation and surgical volume predict survival in head and neck cancer patients receiving surgery. Laryngoscope. 2021;131:E479–88. https://doi.org/10.1002/lary.28744 .

Fernández-Domínguez JC, De Pedro-Gómez JE, Jiménez-López R, Romero-Franco N, Bays Moneo AB, Oliva-Pascual-Vaca Á, et al. Physiotherapists’ evidence-based practice profiles by HS-EBP questionnaire in Spain: a cross-sectional normative study. PLoS ONE. 2022;17(6): e0269460. https://doi.org/10.1371/journal.pone.0269460 .

Fields D, Knudsen HK, Roman PM. Implementation of network for the improvement of addiction treatment (NIATx) processes in substance use disorder treatment centers. J Behav Health Serv Res. 2016;43:354–65. https://doi.org/10.1007/s11414-015-9466-7 .

Fong ZV, Chang DC, Hur C, et al. Variation in long-term oncologic outcomes by type of cancer center accreditation: an analysis of a SEER-Medicare population with pancreatic cancer. Am J Surg. 2020;220(1):29–34. https://doi.org/10.1016/j.amjsurg.2020.03.035 .

García-Romero A, Escribano Á, Tribó JA. The impact of health research on length of stay in Spanish public hospitals. Res Policy. 2017;46:591–604. https://doi.org/10.1016/j.respol.2017.01.006 .

Gilbert GH, Gordan VV, Funkhouser EM, Rindal DB, Fellows JL, Qvist V, et al. Caries treatment in a dental practice-based research network: movement toward stated evidence-based treatment. Community Dent Oral Epidemiol. 2013;41:143–53. https://doi.org/10.1111/cdoe.12008 .

Jonker L, Fisher SJ. NHS Trusts’ clinical research activity and overall CQC performance—Is there a correlation? Public Health. 2015;129:1491–5. https://doi.org/10.1016/j.puhe.2015.07.026 .

Jonker L, Fisher SJ. The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings: a retrospective cross-sectional study. Public Health. 2018;157:1–6. https://doi.org/10.1016/j.puhe.2017.12.022 .

Jonker L, Fisher SJ, Badgett RG. Relationship between staff thriving, through engagement and research activity, and hospital-related outcome measures: a retrospective cross-sectional study. J Healthc Qual Res. 2021;36(3):128–35.

Kirby K, Brandling J, Robinson M, Thomas M, Voss S, Benger J. The experiences of EMS providers taking part in a large randomised trial of airway management during out of hospital cardiac arrest, and the impact on their views and practice. Results of a survey and telephone interviews. Resuscitation. 2020;149:1–9. https://doi.org/10.1016/j.resuscitation.2020.01.034 .

Kirwan CC, Al Sarakbi W, Loncaster J, Chan HY, Thompson AM, Wishart GC. Tumour bed clip localisation for targeted breast radiotherapy: compliance is proportional to trial-related research activity: tumour bed clip localisation in breast radiotherapy. Eur J Surg Oncol. 2014;40:158–62. https://doi.org/10.1016/j.ejso.2013.11.016 .

Levan JM, Brion LP, Wrage LA, Gantz MG, Wyckoff MH, Sánchez PJ, et al. Change in practice after the surfactant, positive pressure and oxygenation randomised trial. Arch Dis Child Fetal Neonatal Ed. 2014;99:F386–90. https://doi.org/10.1136/archdischild-2014-306057 .

Lin DJ, McConkey CC, Nankivell P, Dunn J, Mehanna H. The impact of institutional clinical trial recruitment versus hospital volume on survival outcomes of patients with head and neck cancer: an analysis of the PET-NECK trial outcomes, UKCRN portfolio, and Hospital Episode Statistics (HES) in England. Oral Oncol. 2018;85:40–3. https://doi.org/10.1016/j.oraloncology.2018.08.006 .

Litjens RJ, Oude Rengerink K, Danhof NA, Kruitwagen RF, Mol BW. Does recruitment for multicenter clinical trials improve dissemination and timely implementation of their results? A survey study from the Netherlands. Clin Trials. 2013;10:915–23. https://doi.org/10.1177/1740774513504150 .

Ljunggren M, Weibull CE, Rosander E, et al. Hospital factors and metastatic surgery in colorectal cancer patients, a population-based cohort study. BMC Cancer. 2022;22:907. https://doi.org/10.1186/s12885-022-10005-8 .

Manes E, Tchetchik A, Tobol Y, Durst R, Chodick G. An empirical investigation of “physician congestion” in US University Hospitals. Int J Environ Res Public Health. 2019;16:761. https://doi.org/10.3390/ijerph16050761 .

Marmor S, Begun J, Abraham J, et al. The impact of center accreditation on hematopoietic cell transplantation (HCT). Bone Marrow Transplant. 2015;50:87–94. https://doi.org/10.1038/bmt.2014.219 .

McBride R, Leroux B, Lindblad A, Williams OD, Lehmann M, Rindal DB, et al. Measuring the impact of practice-based research networks on member dentists in the Collaboration on Networked Dental and Oral Health Research, CONDOR. J Dent. 2013;41:393–403. https://doi.org/10.1016/j.jdent.2013.03.005 .

McCarthy FH, Groeneveld PW, Kobrin D, Mcdermott KM, Wirtalla C, Desai ND. Effect of clinical trial experience on transcatheter aortic valve replacement outcomes. Circ Cardiovasc Interv. 2015;8: e002234. https://doi.org/10.1161/CIRCINTERVENTIONS.114.002234 .

McDaniels-Davidson C, Feng CH, Martinez ME, Canchola AJ, Gomez SL, Nodora JN, et al. Improved survival in cervical cancer patients receiving care at National Cancer Institute—Designated cancer centers. Cancer. 2022;128:3479–86. https://doi.org/10.1002/cncr.34404 .

Merkow RP, Chung JW, Paruch JL, Bentrem DJ. Center accreditation and performance on publicly reported quality measures. Ann Surg. 2014;259(6):1091–7. https://doi.org/10.1097/SLA.0000000000000542 .

Meyer AM, Reeder-Hayes KE, Liu H, Wheeler SB, Penn D, Weiner BJ, et al. Differential receipt of sentinel lymph node biopsy within practice-based research networks. Med Care. 2013;51:812–8. https://doi.org/10.1097/MLR.0b013e31829c8ca4 .

Mold JW, Aspy CB, Smith PD, Zink T, Knox L, Lipman PD, et al. Leveraging practice-based research networks to accelerate implementation and diffusion of chronic kidney disease guidelines in primary care practices: a prospective cohort study. Implement Sci. 2014;9:169. https://doi.org/10.1186/s13012-014-0169-x .

Murimwa GZ, Karalis JD, Meier J, et al. Hospital designations and their impact on guideline-concordant care and survival in pancreatic cancer. Do they matter? Ann Surg Oncol. 2023;30:4377–87. https://doi.org/10.1245/s10434-023-13308-7 .

Ozdemir BA, Karthikesalingam A, Sinha S, Poloniecki JD, Hinchliffe RJ, Thompson MM, et al. Research activity and the association with mortality. PLoS ONE. 2015;10: e0118253. https://doi.org/10.1371/journal.pone.0118253 .

Paulson EC, Mitra N, Sonnad S, Armstrong K, Wirtalla C, Kelz RR, et al. National cancer institute designation predicts improved outcomes in colorectal cancer surgery. Ann Surg. 2008;248:675–86. https://doi.org/10.1097/SLA.0b013e318187a757 .

Penn DC, Chang Y, Meyer AM, Defilippo Mack C, Sanoff HK, Stitzenberg KB, et al. Provider-based research networks may improve early access to innovative colon cancer treatment for African Americans treated in the community. Cancer. 2015;121:93–101. https://doi.org/10.1002/cncr.29028 .

Prendergast EA, Perkins S, Engel ME, Cupido B, Francis V, Joachim A, et al. Participation in research improves overall patient management: insights from the global rheumatic heart disease registry (REMEDY). Cardiovasc J Afr. 2018;29:98–105. https://doi.org/10.5830/CVJA-2017-054 .

Rai A, Nastoupil LJ, Williams JN, Lipscomb J, Ward KC, Howard DH, et al. Patterns of use and survival outcomes of positron emission tomography for initial staging in elderly follicular lymphoma patients. Leuk Lymphoma. 2017;58:1570–80. https://doi.org/10.1080/10428194.2016.1253836 .

Rieckmann TR, Abraham AJ, Bride BE. Implementation of motivational interviewing in substance use disorder treatment: research network participation and organizational compatibility. J Addict Med. 2016;10:402–7. https://doi.org/10.1097/ADM.0000000000000251 .

Rieckmann TR, Abraham AJ, Kovas AE, Mcfarland BH, Roman PM. Impact of research network participation on the adoption of buprenorphine for substance abuse treatment. Addict Behav. 2014;39:889–96. https://doi.org/10.1016/j.addbeh.2014.01.016 .

Rindal DB, Flottemesch TJ, Durand EU, Godlevsky OV, Schmidt AM, Gilbert GH. Practice change toward better adherence to evidence-based treatment of early dental decay in the National Dental PBRN. Implement Sci. 2014;9:177. https://doi.org/10.1186/s13012-014-0177-x .

Robinson TG, Wang X, Durham AC, Ford GA, Liao J, Littlewood S, et al. The National Institute for Health Research Hyperacute Stroke Research Centres and the ENCHANTED trial: the impact of enhanced research infrastructure on trial metrics and patient outcomes. Health Res Policy Syst. 2019;17:19. https://doi.org/10.1186/s12961-019-0417-2 .

Rochon J, Du Bois A, Lange T. Mediation analysis of the relationship between institutional research activity and patient survival. BMC Med Res Methodol. 2014;14:9. https://doi.org/10.1186/1471-2288-14-9 .

Salge TO, Vera A. Hospital innovativeness and organizational performance: evidence from English public acute care. Health Care Manag Rev. 2009;34:54–67. https://doi.org/10.1097/01.HMR.0000342978.84307.80 .

Seaburg LA, Wang AT, West CP, Reed DA, Halvorsen AJ, Engstler G, et al. Associations between resident physicians’ publications and clinical performance during residency training. BMC Med Educ. 2016;16:22. https://doi.org/10.1186/s12909-016-0543-2 .

Shen M, Liang X, Li L, Wu Y, Yang Y, Zingg R. The association of attending physicians’ publications and patients’ readmission rates: evidence from tertiary hospitals in china using a retrospective data analysis. Int J Environ Res Public Health. 2022;19(15):9760. https://doi.org/10.3390/ijerph19159760 .

Siracuse JJ, Goodney PP, Menard MT, Rosenfield K, Van Over M, Hamza T, et al. Participation in a chronic limb threatening ischemia randomized trial is inversely correlated with regional amputation rate in limb threatening ischemia patients. Ann Surg. 2021;274(4):621–6. https://doi.org/10.1097/SLA.0000000000005058 .

Tan HJ, Meyer AM, Kuo TM, Smith AB, Wheeler SB, Carpenter WR, et al. Provider-based research networks and diffusion of surgical technologies among patients with early-stage kidney cancer. Cancer. 2015;121:836–43. https://doi.org/10.1002/cncr.29144 .

Tchetchik A, Grinstein A, Manes E, Shapira D, Durst R. From research to practice: which research strategy contributes more to clinical excellence? Comparing high-volume versus high-quality biomedical research. PLoS ONE. 2015;10: e0129259. https://doi.org/10.1371/journal.pone.0129259 .

Trusson D, Rowley E, Bramley L. A mixed-methods study of challenges and benefits of clinical academic careers for nurses, midwives and allied health professionals. BMJ Open. 2019;9: e030595. https://doi.org/10.1136/bmjopen-2019-030595 .

Tsang Y, Ciurlionis L, Kirby AM, Locke I, Venables K, Yarnold JR, et al. Clinical impact of IMPORT HIGH trial (CRUK/06/003) on breast radiotherapy practices in the United Kingdom. Br J Radiol. 2015;88:20150453. https://doi.org/10.1259/bjr.20150453 .

Tsang JLY, Binnie A, Duan EH, Johnstone J, Heels-Ansdell D, Reeve B. Academic and community ICUs participating in a critical care randomized trial: a comparison of patient characteristics and trial metrics. Crit Care Explor. 2022;4(11): e0794. https://doi.org/10.1097/CCE.0000000000000794 .

Van Der Tuuk K, Koopmans CM, Groen H, Mol BW, Van Pampus MG. Impact of the HYPITAT trial on doctors’ behaviour and prevalence of eclampsia in the Netherlands. BJOG. 2011;118:1658–60. https://doi.org/10.1111/j.1471-0528.2011.03138.x .

Venables K, Tsang Y, Ciurlionis L, Coles CE, Yarnold JR. Does participation in clinical trials influence the implementation of new techniques? A look at changing techniques in breast radiotherapy in the UK. Clin Oncol (R Coll Radiol). 2012;24:e100–5. https://doi.org/10.1016/j.clon.2012.06.010 .

Wolfson JA, Sun CL, Wyatt LP, Hurria A, Bhatia S. Impact of care at comprehensive cancer centers on outcome: results from a population-based study. Cancer. 2015;121(21):3885–93. https://doi.org/10.1002/cncr.29576 .

Wuthrick EJ, Zhang Q, Machtay M, Rosenthal DI, Nguyen-Tan PF, Fortin A, et al. Institutional clinical trial accrual volume and survival of patients with head and neck cancer. J Clin Oncol. 2015;33(2):156–64. https://doi.org/10.1200/JCO.2014.56.5218 .

Yawn BP, Pace W, Dietrich A, Bertram S, Kurland M, Graham D, et al. Practice benefit from participating in a practice-based research network study of postpartum depression: a national research network (NRN) report. J Am Board Fam Med. 2010;23:455–64. https://doi.org/10.3122/jabfm.2010.04.090246 .

Young C, Gunasekera H, Kong K, Purcell A, Muthayya S, Vincent F, et al. A case study of enhanced clinical care enabled by Aboriginal health research: the Hearing, EAr health and Language Services (HEALS) project. Aust N Z J Public Health. 2016;40:523–8. https://doi.org/10.1111/1753-6405.12586 .

Rogers E. Diffusion of innovations. 5th ed. New York: Free Press; 2003.

Cohen WM, Levinthal DA. Absorptive capacity: a new perspective on learning and innovation. Adm Sci Q. 1990;35(1):128–52.

Zahra SA, George G. Absorptive capacity: a review, reconceptualization, and extension. Acad Manag Rev. 2002;27:185–203. https://doi.org/10.5465/AMR.2002.6587995 .

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50 .

Roman PM, Abraham AJ, Rothrauff TC, Knudsen HK. A longitudinal study of organizational formation, innovation adoption, and dissemination activities within the national drug abuse treatment clinical trials network. J Subst Abuse Treat. 2010;38(Suppl 1):S44-52. https://doi.org/10.1016/j.jsat.2009.12.008 .

Minasian LM, Carpenter WR, Weiner BJ, Anderson DE, McCaskill-Stevens W, Nelson S, et al. Translating research into evidence-based practice: the National Cancer Institute Community Clinical Oncology Program. Cancer. 2010;116:4440–9. https://doi.org/10.1002/cncr.25248 .

Fennell M, Warneke R. The diffusion of medical innovations: an applied network analysis: environment, development, and public policy and social services. Boston: Springer; 1988. https://doi.org/10.1007/978-1-4684-5436-9 .

Book   Google Scholar  

Peckham S, Eida T, Hashem F, Kendall S. Research engagement and research capacity building: a priority for healthcare organisations in the UK. J Health Organ Manag. 2023;37(3):343–59. https://doi.org/10.1108/JHOM-12-2021-0436 .

Edelman A, Brown A, Pain T, Larkins S, Harvey G. Evaluating research investment and impact at a regional Australian hospital and health service: a programme theory and conceptual framework. Health Res Policy Sys. 2020;18:30. https://doi.org/10.1186/s12961-020-0542-y .

Brown A, Edelman A, Pain T, Larkins S, Harvey G. “We’re not providing the best care if we are not on the cutting edge of research”: a research impact evaluation at a regional Australian hospital and health service. Int J Health Policy Manag. 2022;11(12):3000–11. https://doi.org/10.34172/IJHPM.2022.6529 .

San Francisco declaration on research assessment. https://sfdora.org/ . Accessed 18 Apr 2024.

Ailawadhi S, Advani P, Yang D, Ghosh R, Swaika A, et al. Impact of access to NCI- and NCCN-designated cancer centers on outcomes for multiple myeloma patients: a SEER registry analysis. Cancer. 2016;122:618–25. https://doi.org/10.1002/cncr.29771 .

Tsilimigras DI, Hyer JM, Diaz A, Moris D, Abbas A, Dillhoff M, et al. Impact of cancer center accreditation on outcomes of patients undergoing resection for hepatocellular carcinoma: a SEER-Medicare analysis. Am J Surg. 2021;222(3):570–6. https://doi.org/10.1016/j.amjsurg.2021.01.015 .

Grant J, Buxton MJ. Economic returns to medical research funding. BMJ Open. 2018;8: e022131. https://doi.org/10.1136/bmjopen-2018-022131 .

Schlick CJ, Yang AD. Is there value in cancer center accreditation? Am J Surg. 2023;220(1):27–8. https://doi.org/10.1016/j.amjsurg.2020.05.001 .

Hussein M, Pavlova M, Ghalwash M, Groot W. The impact of hospital accreditation on the quality of healthcare: a systematic literature review. BMC Health Serv Res. 2021;21:1057.

Yoong SL, Bolsewicz K, Reilly K, et al. Describing the evidence-base for research engagement by health care providers and health care organisations: a scoping review. BMC Health Serv Res. 2023;23:75. https://doi.org/10.1186/s12913-022-08887-2 .

Soper B, Yaqub O, Hinrichs S, Marjanovich S, Drabble S, Hanney S, et al. CLAHRCs in practice: combined knowledge transfer and exchange strategies, cultural change, and experimentation. J Health Serv Res Policy. 2013;18:53–64. https://doi.org/10.1177/1355819613499903 .

Kislov R, Wilson PM, Knowles S, Boaden R. Learning from the emergence of NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs): a systematic review of evaluations. Implement Sci. 2018;13:111. https://doi.org/10.1186/s13012-018-0805-y .

Download references

Acknowledgements

The authors would like to thank Teresa Jones for her expert advice on the search strategy. The review was completed as partial fulfilment of Belinda Goodenough’s Masters dissertation at King’s College London. In the original full report for our first review [ 3 ], we gratefully acknowledged the valuable help we had received from our expert advisory group. The members included two patient representatives who were consulted at various stages throughout the project, especially around the necessity of having our systematic review focus on the complexities of benefits from research engagement by healthcare organizations and staff, while separate reviews and analyses focussed on the benefits of PPI in health research.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, King’s Policy Institute, King’s College London, Virginia Woolf Building, 20 Kingsway, London, United Kingdom

Annette Boaz

The Sax Institute, Sydney, NSW, Australia

Belinda Goodenough

Brunel University London, Uxbridge, United Kingdom

Stephen Hanney & Bryony Soper

You can also search for this author in PubMed   Google Scholar

Contributions

All authors were involved in planning the study. B.G. conducted the searches, with all authors involved in screening and analysis. B.G. produced an initial draft of the paper. The final version of the paper was produced collaboratively by all the authors.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Boaz, A., Goodenough, B., Hanney, S. et al. If health organisations and staff engage in research, does healthcare improve? Strengthening the evidence base through systematic reviews. Health Res Policy Sys 22 , 113 (2024). https://doi.org/10.1186/s12961-024-01187-7

Download citation

Received : 14 May 2024

Accepted : 22 July 2024

Published : 19 August 2024

DOI : https://doi.org/10.1186/s12961-024-01187-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Health Research Policy and Systems

ISSN: 1478-4505

research paper on performance management system

American Psychological Association

References provide the information necessary for readers to identify and retrieve each work cited in the text .

Check each reference carefully against the original publication to ensure information is accurate and complete. Accurately prepared references help establish your credibility as a careful researcher and writer.

Consistency in reference formatting allows readers to focus on the content of your reference list, discerning both the types of works you consulted and the important reference elements (who, when, what, and where) with ease. When you present each reference in a consistent fashion, readers do not need to spend time determining how you organized the information. And when searching the literature yourself, you also save time and effort when reading reference lists in the works of others that are written in APA Style.

research paper on performance management system

Academic Writer ®

Master academic writing with APA’s essential teaching and learning resource

illustration or abstract figure and computer screen

Course Adoption

Teaching APA Style? Become a course adopter of the 7th edition Publication Manual

illustration of woman using a pencil to point to text on a clipboard

Instructional Aids

Guides, checklists, webinars, tutorials, and sample papers for anyone looking to improve their knowledge of APA Style

Identification of avoidable patients at triage in a Paediatric Emergency Department: a decision support system using predictive analytics

BMC Emergency Medicine volume  24 , Article number:  149 ( 2024 ) Cite this article

11 Accesses

Metrics details

Crowding has been a longstanding issue in emergency departments. To address this, a fast-track system for avoidable patients is being implemented in the Paediatric Emergency Department where our study is conducted. Our goal is to develop an optimized Decision Support System that helps in directing patients to this fast track. We evaluated various Machine Learning models, focusing on a balance between complexity, predictive performance, and interpretability.

This is a retrospective study considering all visits to a university-affiliated metropolitan hospital’s PED between 2014 and 2019. Using information available at the time of triage, we trained several models to predict whether a visit is avoidable and should be directed to a fast-track area.

A total of 507,708 visits to the PED were used in the training and testing of the models. Regarding the outcome, 41.6% of the visits were considered avoidable. Except for the classification made by triage rules, i.e. considering levels 1,2, and 3 as non-avoidable and 4 and 5 as avoidable, all models had similar results in model’s evaluation metrics, e.g. Area Under the Curve ranging from 74% to 80%.

Conclusions

Regarding predictive performance, the pruned decision tree had evaluation metrics results that were comparable to the other ML models. Furthermore, it offers a low complexity and easy to implement solution. When considering interpretability, a paramount requisite in healthcare since it relates to the trustworthiness and transparency of the system, the pruned decision tree excels.

Overall, this paper contributes to the growing body of research on the use of machine learning in healthcare. It highlights practical benefits for patients and healthcare systems of the use ML-based DSS in emergency medicine. Moreover, the obtained results can potentially help to design patients’ flow management strategies in PED settings, which has been sought as a solution for addressing the long-standing problem of overcrowding.

Peer Review reports

Introduction

Emergency Department (ED) crowding occurs when demands are greater than the hospitals’ capacity to ensure timely care in the ED. This is a multifactorial problem with multiple solutions. These problems can be tackled by influencing demand e.g., implementing gatekeeping policies, or optimizing the service provided. Furthermore, ED overcrowding and high patient volumes can result in delays in care, suboptimal treatment decisions, and increased risk of adverse events, including mortality, both in paediatric and adult settings [ 1 , 2 , 3 ]. To address these challenges, various approaches have been proposed. One of the most studied is the alteration of patient flow i.e., the implementation of fast tracks, based on the severity of the patients’ condition. This approach have shown to be able to improve both efficiency and outcomes, as well as reducing waiting times and overcrowding [ 4 , 5 ].

Accessing the patients’ severity i.e. the identification of clinically divertible attendances or clinically unnecessary attendances patients [ 6 ], to be steered to fast tracks, has been made mostly using triage levels [ 7 , 8 , 9 , 10 ] or empirical rules [ 11 ]. The validity and accuracy of the identification of these avoidable visits is paramount [ 12 ], since it evolves not only patient flow efficiency but also patient safety. On the other hand, the emergence of robust machine learning (ML) algorithms has shown potential to improve predictive ability of various outcomes [ 13 , 14 , 15 , 16 ], which in turn could be leveraged to aid in the identification of these avoidable visits [ 16 , 17 ].

In the hospital this study is being conducted, the Paediatric Emergency Department (PED) is undergoing restructuring, which includes the introduction of a fast-track system. Consequently, there’s a need to develop an algorithm to identify patients who are suitable for this expedited care pathway. A system that identifies these patients accurately has the potential to reduce the amount of time patients spend in the ED, reducing departmental crowding and ultimately support better patient outcomes. It will also, in all likelihood, help the reduction of ED overcrowding and facilitate a more effective allocation of healthcare resources.

Hence, we aim to create a data-driven, optimized DSS to aid in the selection of patients deemed avoidable and redirect them for a fast track. To achieve this, we compared different ML models and approaches, considering the balance between implementation complexity, predictive performance, and interpretability.

Study design, setting and participants

This study was an observational and retrospective research, conducted in a university-affiliated metropolitan hospital's PED. The hospital serves a population of about 800,000 and receives an average of 76,000 visits annually from an estimated 137,016 children or adolescents aged 0 to 17 years [ 18 ].

In the PED, there are 4–5 physicians and 7–8 nurses, working in 12-h shifts to ensure 24/7 coverage.

In this study, all presentations made to the hospital’s PED (i.e., from 0 to 17 years old) in a 4-year period (between 01/Jan/2016, and 31/Dec/2019) were considered.

The PED nursing team triages visitors according to the Canadian Triage and Acuity Scale paediatric guidelines (PaedCTAS), which is structured around evaluating physiological factors, including appearance, neurological status, respiratory rate, heart rate, and perfusion, alongside presenting symptoms to determine triage levels. Similar to the adult version of the Canadian Triage and Acuity Scale (CTAS), the PaedCTAS delineates five levels of triage i.e. Level 1(Red)—“Resuscitation”, Level 2 (Orange)—“Emergent”, Level 3 (Yellow)—“Urgent”, Level 4 (Green)—“Less Urgent” and Level 5 (Blue)—“Non Urgent”. These levels reflect the severity and urgency of the patient's condition, target times for medical assessment and intervention, and provide examples of typical clinical presentations and critical diagnoses [ 19 ].

This paper follows the structure presented in the RECORD statement i.e. The REporting of studies Conducted using Observational Routinely-collected health Data [ 20 ].

PED restructuring

The hospital's PED is undergoing a significant restructuring.. Along with this restructuring, a fast track for avoidable patients is to be implemented. A schema of the fast-track configuration is presented in Fig.  1 .

figure 1

Schema of the restructuring of the Paediatric Emergency department regarding avoidable patient flow

Data collection

All patient data is registered using a proprietary information system called JOne, where events are logged e.g. attending medical staff, diagnoses and cause of admission. All sociodemographic information and triage procedures are also registered in this information system.

All data access permissions, i.e., from the hospital board of directors, hospital epidemiology centre, information access officer and ethical committee were granted for this study. [FMUP 180/18].

The dependent variable was defined by the conjunction of several PED markers, resulting in a restrictive definition of avoidable. A PED visit is considered avoidable if the patient is discharged home and no diagnostic tests (i.e., blood tests and radiology exams), procedures or medications were required during the stay. The patients were also not asked to stay in the ED, for the physician to better assess the condition’s evolution. In summary, an avoidable visit is done by a patient that only has contact with the physician and is discharged home. The bullet list below summarizes the approach taken in the construction of the outcome to be predicted.

Avoidable visit;

◦ The patient was not medicated

◦ The patient did not undergo any radiologic exams

◦ The patient did not undergo any blood analysis

◦ The patient did not stay for observations

◦ The patient was discharged home

Predictors and feature selection

A list of variables known at the time of the patient’s triage were used as predictors. Feature selection was done on a model-by-model basis, since associations between variables, redundant variables and low variance variables are handled differently depending on the model.

Season – Season of the year of patient’s arrival to the PED

Month – Month of the year of patient’s arrival to the PED

Day of week –weekday of the patient’s arrival to the PED

Hour of day – Time of day of patient’s arrival to the PED, in. hourly slots

Pretriage discriminator group –Paediatric Assessment Triangle group selection as described in PaedCTAS [ 19 ]

Pretriage discriminator –Paediatric Assessment Triangle selection as described in PaedCTAS [ 19 ]

Main complaint group—Triage complaint group as described in PaedCTAS [ 19 ]

Main complaint discriminator – Triage complaint as described in PaedCTAS [ 19 ]

Residence municipality – patient’s residence, municipalities outside the catchment area were residual and were grouped as other

Triage level – PaedCTAS level of triage

Referral – A referral patient was defined as ‘not walk-in’ patient e.g. referral from PCP, private clinic or other hospitals

Made return visit X hours prior- Patient made at least one visit to the PED X (i.e.12, 24, 48, 72) hours prior to current visit i.e. the current visit is a return visit

Visit by frequent attender – Visit made by frequent attender i.e. > 4 visits per year

Machine learning models

Several ML models were created to evaluate the appropriateness for implementation in this context. These models ranged from simple rules, based on triage level (i.e. visits triaged levels 1, 2 and 3 are directed to regular emergency department patient flow while visits triaged levels 4 and 5 are directed to fast track), to a neural network. The appropriateness involved three dimensions: (1) complexity, some models could be implemented as simple rules, others need to be integrated into the hospital’s information system; (2) interpretability, can the reasoning behind the decision be understood i.e., glass box model or not i.e., black box, and (3) predictive performance. All the models used in this study and their classifications are enumerated in Table  1 .

The ML models tested have very different characteristics and some have parameters that are set in advance and control various aspects of the training process itself i.e. hyperparameters. These hyperparameters can be changed, and the performance of the models evaluated in a process called hyperparameter tunning. As an example, and considering decision trees, cost complexity is used for pruning the tree to avoid overfitting. It adjusts the trade-off between accuracy (and possible overfitting) and tree simplicity.

The default threshold of 0.5 for binary classification was kept for all the models.

Data preparation and model training

The dataset was randomly split 20% for testing and 80% for training with stratification for the outcome. On the training set, it was used a 2-fold cross validation with stratification for the outcome, repeated 20 times.

All categorical missing values were imputed the category “unknown”, there were no missing numerical data. Only the variables “Pretriage discriminator” and “Pretriage discriminator group” had an expected, and relevant missing count (96.7%). These variables are only filled in particular circumstances during the triage algorithm. All other categorical variables had negligible (> 0.5%) of missing count.

For tensor flow model and the XGboost model, dummy variables were created i.e., one binary numeric variable was created for each category. Furthermore, the variable “Triage maincomplaint discriminator” was removed from the simple tree model because it has too many categories (217) and would be unusable in the paper-based approach. To mitigate complexly, the tree depth maximum was also set to 5 in hyper parameter tuning phase. The variable “Triage maincomplaint discriminator” was also removed from the logistic regression model, as some levels had only a small number of observations and the information was already aggregated in the variable “Main complaint group”.

All the data analysis was performed using R version 4.2.2 (2022–10-31) [ 23 ]. The integrated development environment (IDE) used was RStudio Version 2022.12.0 + 353 [ 24 ]. The ecosystem of packages “tidymodels” was used. The specific packages used for training the ML models are underlined in Table  1 .

Models’ evaluation

The evaluation metrics used in our study to compare the performance of different machine learning methods are enumerated below. Accuracy (accuracy), Negative Predictive Value (npv), Positive Predictive Value (ppv), Sensitivity (sens) Specificity (spec), Kappa (kap), Area Under Curve (roc_auc) and F-measure (F_meas) that combines ppv and sensitivity, providing a single score that reflects both aspects of a model's performance. Furthermore, False Positives (FP) i.e., non-avoidable visits classified as avoidable, False Negatives (FN) i.e. avoidable visits classified as non-avoidable, True Positives i.e. visits classified correctly as avoidable and True Negatives (TN) i.e. visits classified correctly non-avoidable were also computed for each model.

The dataset utilized for training and testing the models comprised a total of 507,708 visits to the pediatric emergency department. Of these visits 17.4% were referrals, and 4.4% resulted in hospital admissions. Females accounted for 46.8% of the visits. The average age of patients was 7 years, with a standard deviation of 5.5 years. The mean length of stay was 102.4 min, with a standard deviation of 154.5 min. Regarding the outcome, 41.6% of the visits were considered avoidable. Concerning triage levels, 0.2% were triaged level 1, 5.6% level 2, 39.4% level 3, 50.4% level 4 and 4.5% level 5.

Models’ performance

Considering the metrics that evaluate the models globally i.e. accuracy, f_meas, kap and roc_auc, all ML models outperformed the classification made by triage rules i.e. visits triaged levels 1 (red), 2 (orange) and 3 (yellow) were considered non-avoidable and visits triaged levels 4 (green) and 5 (blue) were considered avoidable. Except for the classification made by triage rules, the models had similar performance, namely accuracy, ranging from 70 to 72% and AUCs ranging from 74 to 80%. However, regarding measures that evaluate the specific performance i.e., NPV, PPV, sensitivity and specificity, there is greater variation Moreover, the triage rules model was also outperformed by the all ML models in terms of measures addressing specific performance, except for sensitivity.

Predictions on test data

For a more pragmatic and in-depth analysis and drilling down from Figs.  2  and  3 presents a confusion matrix’s inspired plot, where the model’s classifications are compared. This visualization enables us to calculate, for a particular number of visits, how many are correctly classified and misclassified, for any given model. It is important to highlight the low FP proportion of all ML models and the high TN proportion, with relatively small differences between them. On the other hand, the classification made by triage rules had the highest FP proportion and the lowest TN proportion. The FN rate and the TP rate were relatively constant across all models.

figure 2

Bar plot with error bars, comparing the performance of the predictive models used to classify if visit to the Paediatrics emergency department is avoidable, and should be directed the fast-track area, by metric. Results were obtained from the twofold cross validation with stratification for the outcome, repeated 20 times

figure 3

Percentage of visits, for each model, incorrectly assigned to the fast-track i.e., non-avoidable classified as avoidable (FP), incorrectly assigned to normal flow i.e. avoidable classified as non-avoidable (FN) and percentage of visits classified correctly, either avoidable (TP) or non-avoidable (TN). Data was obtained from the test dataset when the models were fitted

Simple classification tree

When constructing the model for the simple classification tree, the hyperparameter tunning was limited to a tree depth of 5. Fig.  4 shows that above a tree depth of 4 there is no significant improvement in overall performance i.e., accuracy and AUC. Furthermore, when the value of cost complexity decreases, sensitivity increases.

figure 4

Plots with the results from a grid hyperparameter tunning for the simple decision tree. Each plot refers to a specific tree depth. Metric’s mean is the result of a twofold cross validation repeated 20 times

Figure  5 presents a decision tree with the hyperparameters set to: tree depth of 4 and a cost complexity of 1e-4. This combination was chosen for the best balance between sensitivity and specificity, not having a significant impact on overall performance i.e., accuracy and AUC. This particular tree only used the triage’s “Main complaint group” and triage level.

figure 5

Decision tree with the hyperparameters set to tree depth of 4 and a cost complexity of 1e-4, for the classification of visits as avoidable (Yes) or non-avoidable (No), and redirection to a fast track or regular flow respectively, in the Paediatric emergency depart. This tree was created from the test dataset

The major objective of this study was to evaluate several ML models to be implemented in the PED and aid in the decision if a visit to the Paediatrics emergency department is avoidable and should be directed to the fast-track area or is non-avoidable and can stay in the regular flow.

To the best of our knowledge, the assignment of visits to a low acuity fast track is mostly done by the assigned triage levels and there are no studies with a pragmatic focus on implementation with data-driven approaches. [ 7 , 8 , 9 , 10 , 16 , 25 , 26 ].

Our approach was to leverage the power of ML to aid in the assignment for the fast-track. First, creating an outcome based on resource utilization and PED discharge destination. And afterwards trying to predict it using information known at the time of triage.

Summary of main findings

Regarding overall performance metrics, all ML models had similar performance and outperformed the classification made by triage rules. Drilling down, errors made by triage rules were mainly false positives, i.e. non avoidable patients sent to the fast track.

The pruned decision tree performed only slightly worse in the overall metrics than all the other ML models and their errors in classification went in the same direction as the other more complex models, as can be seen when analysing sensitivity, specificity or the confusion matrix.

Results contextualization

This study's results regarding AUC were slightly higher than those found by Chang et al. who made a similar study using ML to identify low-severity patients. However, the definition of candidates for fast track was the time interval between the triage registry and being discharged within less than 4 h. Therefore, depending on the setting, this could be a problem, since the length of stay is greatly influenced by the triage level and the hour of the day. Hence, we think that this study's approach, only considering resource usage and discharge destination, appears less prone to confounders [ 27 ].

In a study by Kwon et al. [ 28 ] where the aim was to identify high-risk patients at the time of triage, the ML algorithm predicted in-hospital mortality, critical care, and hospitalization more accurately than existing triage systems. When considering only hospitalization, the results are very similar to our study [ 28 ]. Despite the different outcomes, these results support the superiority of ML models over triage system’s classification to predict outcomes in the ED. This is further reinforced and expanded by the results of a systematic review by Kareen et al. where it was found that in an ED setting, ML models outperformed usual care in most diagnostic and prognostic predictions across all studies [ 16 ].

It is essential to consider the FP values for each model, i.e., visits that the models classified as avoidable that really were not avoidable. While false negative (FN) proportion, i.e., visits that the models classified as non-avoidable that were avoidable, are directed to the previous patient flow, a high FP proportion might be a reason for concern since they are non-avoidable visits directed to the fast track. In this regard, the split made by triage rules is a reason for worry, since its FP proportion doubles the highest ML model. This highlights the necessity to include all relevant stakeholders (e.g. physicians, nurses, administrative personnel and patients) in the development of the DSS and is vital to its successful implementation, as only these stakeholders know the particularities of the context where the DSS is to be deployed. The ML models should be evaluated considering not only the performance metrics but also potential clinical consequences and impact on patient outcomes [ 16 , 29 ].

Implications for policy and practice

To discuss the choice of the model to be implemented is necessary to return to the three evaluation dimensions stated in the introduction: complexity, predictive performance, and interpretability. The pruned decision tree is very simple to implement. Based on two or three simple rules, a nurse at the end of triage could redirect the patient with minimal increased logistic burden. As a consequence of its simplicity, the time to implementation and cost is negligible. Regarding the performance assigning patients to a fast track based on triage levels is clearly not the best solution, given the high proportion of FP. Despite not being the best model, the pruned decision tree is outperformed by other more complex models but only by a few percentage points. The last dimension is extremely important and goes beyond interpretation, it relates the understanding to how the decision was made, and if the explanation is satisfactory, it builds trust [ 30 ]. And a trustworthy and explainable system has been a stakeholder requisite, especially in healthcare [ 31 , 32 ]. In this regard, rules extracted from a low-depth decision tree excels.

The choice of the model was clear. The pruned decision tree could be implemented immediately, and the patients and staff could immediately reap the rewards of an improved PED workflow. The resources committed to the implementation are low, hence can be easily replaced if a better alternative is developed.

Limitations

The definition of avoidable visit was chosen among others [ 6 , 33 , 34 ] for being triage system agnostic, better reflecting the visit’s lack of necessity for the hospital’s resources. Nevertheless, the multiple definitions used in the field make comparisons less accurate.

There are few clinical parameters available for model building. Nevertheless, if more variables were to be collected and used, the model's performance could never worsen.

The data gathered had the original purpose of providing care to the patients in the PED, therefore subjected to the bias of any observational study based on routinely collected data, e.g. information system downtime and the inability to control how the variables are collected.

The deployment phase will present challenges and model’s suggestions might have to be adapted the day-to-day operations and decision-making processes in clinical settings.

External validity of the models was not tested. However, given the intended use of the models, it's crucial to consider both the target population and the setting. Thus, if this study were to be conducted in different settings or with different populations, the models would need to be retrained with the data available [ 35 ].

This study demonstrates the substantial potential of ML models to enhance decision-making processes in Emergency Departments, regarding the assignment of patients to appropriate care paths. Our findings underscore the superior performance of ML models over more traditional methods in determining the patient flow. While all tested ML models performed well, the pruned decision tree model emerged as a practical choice due to its simplicity, ease of implementation, and relatively high accuracy. Moreover, this model supports the need for interpretable and trustworthy systems in healthcare, as it allows healthcare providers to understand and trust the basis of its predictions. Moving forward, the integration of ML into clinical settings should continue to focus on balancing complexity, predictive performance, and interpretability, ensuring that such tools are not only technically effective but also align with the practical realities and ethical considerations of medical practice. Finally, the obtained results can potentially help to design patients’ flow management strategies in PED settings, which has been sought as a solution for addressing the long-standing problem of overcrowding.

More data-driven approaches, where the patient and healthcare professionals are put first and technology serves an instrumental role in solving the problem, are necessary in this age of AI hype. Small, targeted interventions to solve real-world problems with real-world data are paramount to the future of healthcare.

Availability of data and materials

The data that support the findings of this study are available from the hospital, but restrictions apply to the availability of these data, which were used under hospital’s authorization for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and authorized by the hospital’s administration.

Abbreviations

Emergency Department

Decision Support Systems

Electronic Health Records

Paediatric Emergency Department

Hospital's Information System

Comma Separated Values

Negative Predictive Value

Positive Predictive Value

Sensitivity

Specificity

Area Under Curve

Canadian Triage and Acuity Scale paediatric guidelines

True Negative Rate

True Positive Rate

True Positive

True Negative

False Positive

H. R. Rasouli et al., “Outcomes of Crowding in Emergency Departments; a Systematic Review,” Arch. Acad. Emerg. Med., vol. 7, no. 1, Aug. 2019. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6785211/ . Accessed: Apr. 30, 2021.

Chan M, Meckler G, Doan Q. Paediatric emergency department overcrowding and adverse patient outcomes. Paediatr Child Health. 2017;22(7):377–81. https://doi.org/10.1093/pch/pxx111 .

Article   PubMed   PubMed Central   Google Scholar  

Doan Q, et al. The impact of pediatric emergency department crowding on patient and health care system outcomes: a multicentre cohort study. CMAJ Can Med Assoc J J Assoc Med Can. 2019;191(23):E627–35. https://doi.org/10.1503/cmaj.181426 .

Article   Google Scholar  

Wiler JL, et al. Optimizing emergency department front-end operations. Ann Emerg Med. 2010;55(2):142–160.e1. https://doi.org/10.1016/j.annemergmed.2009.05.021 .

Article   PubMed   Google Scholar  

Oredsson S, et al. A systematic review of triage-related interventions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med. 2011;19:43. https://doi.org/10.1186/1757-7241-19-43 .

Parkinson B, Meacock R, Checkland K, Sutton M. Clarifying the concept of avoidable emergency department attendance. J Health Serv Res Policy. 2021;26(1):68–73. https://doi.org/10.1177/1355819620921894 .

Arya R, Wei G, McCoy JV, Crane J, Ohman-Strickland P, Eisenstein RM. Reasing length of stay in the emergency department with a split emergency severity index 3 patient flow model. Acad Emerg Med. 2013;20(11):1171–9. https://doi.org/10.1111/acem.12249 .

Kwa P, Blake DF. Fast track: has it changed patient care in the emergency department? Emerg Med Australas. 2008;20(1):10–5. https://doi.org/10.1111/j.1742-6723.2007.01021.x .

Sanchez M, Smally AJ, Grant RJ, Jacobs LM. Effects of a fast-track area on emergency department performance. J Emerg Med. 2006;31(1):117–20. https://doi.org/10.1016/j.jemermed.2005.08.019 .

O’Brien D, Williams A, Blondell K, Jelinek GA. Impact of streaming ‘fast track’ emergency department patients. Aust Health Rev. 2006;30(4):525–32. https://doi.org/10.1071/ah060525 .

Martin HA, Noble M, Wilmarth J. Improving patient flow and decreasing patient length of stay in the Pediatric Emergency Department through implementation of a fast track. Adv Emerg Nurs J. 2021;43(2):162–9. https://doi.org/10.1097/TME.0000000000000351 .

Vance J, Sprivulis P. Triage nurses validly and reliably estimate emergency department patient complexity. Emerg Med Australas. 2005;17(4):382–6. https://doi.org/10.1111/j.1742-6723.2005.00761.x .

Feretzakis G, et al. Using machine learning techniques to predict hospital admission at the emergency department. J Crit Care Med. 2022;8(2):107–16. https://doi.org/10.2478/jccm-2022-0003 .

Miles J, Turner J, Jacques R, Williams J, Mason S. Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: a systematic review. Diagn Progn Res. 2020;4:16. https://doi.org/10.1186/s41512-020-00084-1 .

Mueller B, Kinoshita T, Peebles A, Graber MA, Lee S. Artificial intelligence and machine learning in emergency medicine: a narrative review. Acute Med Surg. 2022;9(1):e740. https://doi.org/10.1002/ams2.740 .

Kareemi H, Vaillancourt C, Rosenberg H, Fournier K, Yadav K. Machine learning versus usual care for diagnostic and prognostic prediction in the emergency department: a systematic review. Acad Emerg Med. 2021;28(2):184–96. https://doi.org/10.1111/acem.14190 .

Raita Y, Goto T, Faridi MK, Brown DFM, Camargo CA, Hasegawa K. Emergency department triage prediction of clinical outcomes using machine learning models. Crit Care. 2019;23:64. https://doi.org/10.1186/s13054-019-2351-7 .

“Statistics Portugal. Instituto Nacional de Estatística - Portugal.” Accessed: Feb. 26, 2020. Available: https://www.ine.pt

Warren DW, et al. Revisions to the Canadian Triage and Acuity Scale paediatric guidelines (PaedCTAS). CJEM. 2008;10(3):224–43.

Benchimol EI, et al. The Reporting of studies conducted using observational routinely-collected health data (RECORD) statement. PLOS Med. O. 2015;12(10):e1001885. https://doi.org/10.1371/journal.pmed.1001885 .

Chen T, Guestrin C. “XGBoost: a scalable tree boosting system,” in proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, in KDD ’16. New York, USA: Association for Computing Machinery; 2016. p. 785–94. https://doi.org/10.1145/2939672.2939785 .

Book   Google Scholar  

Arik SÖ, Pfister T. “TabNet: attentive interpretable tabular learning,” Proc. AAAI Conf. Artif. Intell. 2021;35(8). https://doi.org/10.1609/aaai.v35i8.16826 .

“R: The R Project for Statistical Computing.” Accessed: Aug. 31, 2021. Available: https://www.r-project.org/

“RStudio | Open source & professional software for data science teams.” Accessed: Aug. 31, 2021. Available: https://rstudio.com/

Sánchez-Salmerón R, et al. Machine learning methods applied to triage in emergency services: a systematic review. Int Emerg Nurs. 2022;60:101109. https://doi.org/10.1016/j.ienj.2021.101109 .

Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JMC. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med. J. 2020;102:101762.  https://doi.org/10.1016/j.artmed.2019.101762 .

Chang Y-H, et al. Machine learning–based triage to identify low-severity patients with a short discharge length of stay in emergency department. BMC Emerg Med. 2022;22(1):88. https://doi.org/10.1186/s12873-022-00632-6 .

Kwon J, Lee Y, Lee Y, Lee S, Park H, Park J. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS ONE. 2018;13(10):e0205836.  https://doi.org/10.1371/journal.pone.0205836 .

Article   PubMed   PubMed Central   CAS   Google Scholar  

F. S. van Royen, F. W. Asselbergs, F. Alfonso, P. Vardas, and M. van Smeden, “Five critical quality criteria for artificial intelligence-based prediction models,” Eur. Heart J., p. ehad727, Oct. 2023, https://doi.org/10.1093/eurheartj/ehad727 .

P. Madumal, T. Miller, L. Sonenberg, and F. Vetere, “Explainable Reinforcement Learning Through a Causal Lens.” arXiv, Nov. 20, 2019. https://doi.org/10.48550/arXiv.1905.10958 .

Combi C, et al. A manifesto on explainability for artificial intelligence in medicine. Artif Intell Med. N. 2022;133:102423.  https://doi.org/10.1016/j.artmed.2022.102423 .

Langer M, et al. What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artif Intell. J. 2021;296:103473.  https://doi.org/10.1016/j.artint.2021.103473 .

Mistry RD, Brousseau DC, Alessandrini EA. Urgency classification methods for emergency department visits: do they measure up? Pediatr Emerg Care. 2008;24(12):870–4. https://doi.org/10.1097/PEC.0b013e31818fa79d .

Hsia RY, Niedzwiecki M. Avoidable emergency department visits: a starting point. Int J Qual Health Care. 2017;29(5):642–5. https://doi.org/10.1093/intqhc/mzx081 .

Sperrin M, Riley RD, Collins GS, Martin GP. Targeted validation: validating clinical prediction models in their intended population and setting. Diagn Progn Res. 2022;6(1):24. https://doi.org/10.1186/s41512-022-00136-8 .

Download references

Acknowledgements

João Viana would like to thank “Fundação para a Ciência e Tecnologia (FCT)” for the funding of his PhD research.

João Viana was funded by “Fundação para a Ciência e Tecnologia (FCT)”, Portugal under PhD grant number PD/BD/129833/2018. The remaining authors have no financial relationships relevant to this article to disclose. This article was supported by National Funds through FCT – Fundação para a Ciência e a Tecnologia , I.P., within Cintesis, R&D Unit (reference UIDB/4255/2020).

Author information

Authors and affiliations.

CINTESIS – Centre for Health Technology and Services Research, University of Porto, Porto, Portugal

João Viana, Júlio Souza & Alberto Freitas

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine of the University of Porto Al. Prof. Hernâni Monteiro, Porto, 4200 - 319, Portugal

João Viana & Alberto Freitas

Institute of Engineering – Polytechnic of Porto, Porto, Portugal

Júlio Souza

Serviço de Pediatria / Urgência Pediátrica, UAG da Mulher E da Criança, Centro Hospitalar Universitário de São João, Porto, Portugal

Ruben Rocha & Almeida Santos

Departamento de Ginecologia-Obstetrícia e Pediatria, Faculty of Medicine of the University of Porto, Porto, Portugal

Almeida Santos

You can also search for this author in PubMed   Google Scholar

Contributions

JV mainly contributed to the drafting of the introduction, results, discussion and study design. RR and AS attested to the clinical and managerial appropriateness of the analysis. AF and JS critically reviewed the manuscript for important intellectual content. All authors contributed to the study design, reviewed, and approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to João Viana .

Ethics declarations

Ethics approval and consent to participate.

The ethics committee of “Centro Hospitalar Universitário de São João” ([email protected]) granted authorization for this study and assigned the number 180/18. All the methods were carried out in accordance with the relevant guidelines and regulations. The data was anonymized, and the informed consent was waived by an institutional review board “Gabinete do Responsável pelo Acesso à Informação (RAI)” ([email protected]) for all the participants according to the hospital’s administration’s regulations for research on secondary data.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Viana, J., Souza, J., Rocha, R. et al. Identification of avoidable patients at triage in a Paediatric Emergency Department: a decision support system using predictive analytics. BMC Emerg Med 24 , 149 (2024). https://doi.org/10.1186/s12873-024-01029-3

Download citation

Received : 06 November 2023

Accepted : 20 June 2024

Published : 18 August 2024

DOI : https://doi.org/10.1186/s12873-024-01029-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

BMC Emergency Medicine

ISSN: 1471-227X

research paper on performance management system

Information

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

buildings-logo

Article Menu

research paper on performance management system

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Enhancing electric vehicle charging infrastructure: a techno-economic analysis of distributed energy resources and local grid integration.

research paper on performance management system

1. Introduction

2. related work, 2.1. studies on infrastructure design of evcs, 2.2. studies focusing on optimization strategies for ev charging, 2.3. studies focusing on service fee calculations for ev charging, 2.4. studies focusing on the integration of ders and evcss, 3. process for evcs techno-economic analysis, 3.1. selection of evcs testbeds and analysis of building types using public data, 3.2. examination of the operational characteristics of evcs testbeds by building type, 3.2.1. monthly operation characteristics analysis, 3.2.2. weekly operation characterization, 3.2.3. hourly operational characterization, 4. techno-economic analysis for evcs based on local grid, 4.1. examination of revenue generated by the evcs charging services, 4.2. examination of the cost of electricity used for evcs fast charging service, 4.3. examination of roi investment process based on evcs profit, 4.4. technical and economic analysis of local grid operation to evcs testbeds, 5. technical and economic analysis of evcss based on integration of ders and local grid operations, 5.1. simulation of ders utilizing pv, 5.2. technical and economic analysis of evcs testbeds integrated with ders and local power grid operations, 6. conclusions and discussion, author contributions, data availability statement, conflicts of interest.

Click here to enlarge figure

Type202020212022
Data Collection PeriodJanuary 2020~
December 2020
January 2021~
December 2021
January 2022~
December 2022
Number of EVCSs
(EA)
391409443
Amount of Collected Data
(Sessions)
278,071138,999252,149
TypeResidential
Facilities
Educational/Cultural
Facilities
Neighborhood
Facilities
Commercial/Business
Facilities
Public
Facilities
Other
Facilities
Number of
EVCSs
(EA)
28110618706
Number of
Fast Chargers
(EA)
3012113351018
EVCS
Charging
Session (times)
31,1945936932118,72567,0413939
EVCS
Charging Amount
(kWh)
780,323152,814221,760437,5451,609,89198,662
Charging Time
(min)
2,314,955322,653447,157947,5663,487,964223,283
Average Charge Time per Session
(min)
745448515257
Average Charging Session
(Times)
8516255113811
Type
(Unit: USD/kWh)
Demand PriceEnergy Charge
TOU ElectricitySummerSpring,
Autumn
Winter
Tariff
(EVCS)
1.87Low load0.0450.0390.052
Medium load0.0740.0460.066
Peak load0.090.0490.076
Type
(Unit: USD)
Residential Facility
Testbed
Commercial/
Business Facility
Testbed
Public Facility
Testbed
Total Investment
(Fast charger, 1EA)
14,484.514,484.514,484.5
EVCS Revenue3760.39832.511,587.5
Demand Charge1121.11121.11121.1
Total Energy Charge1131.23365.23741.3
EVCS Profit15085346.16725.1
ROI (%)10.4%36.9%46.4%
EVCS Payback Period (years)9.612.712.15
Type (Unit: m )Residential FacilityCommercial/
Business Facility
Public Facility
Area640570608
Operational PeriodResidential Facility
(Unit: USD)
Commercial/
Business Facility
(Unit: USD)
Public Facility
(Unit: USD)
Jan 2020481.7284.5336.3
Feb 2020475.5326.8313.3
Mar 2020671.1495.8566.8
Apr 2020817.6697.3676.9
May 2020774.8657.8665.8
Jun 2020674.4522.5515.5
Jul 2020570.2405.7369.5
Aug 2020554.8352.4304.0
Sep 2020633.3519.8445.4
Oct 2020734.5568.8550.2
Nov 2020544.7403.1340.5
Dec 2020572.3397.1361.6
total7504.85631.65445.7
Type
(Unit: USD)
Residential Facility
Testbed
Commercial/
Business Facility
Testbed
Public Facility
Testbed
Total Investment
(Fast charger, 1EA)
14,484.514,484.514,484.5
EVCS Revenue3760.39832.511,587.5
Demand Charge1121.11121.11121.1
Total Energy Charge68822852555.6
Residual Energy
Sales Revenue
7504.85631.65445.7
EVCS Profit945612,05813,356.5
ROI (%)65.383.292.2
EVCS Payback Period (years)1.531.201.08
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Lee, T.; Yoon, G.; Kang, B.; Choi, M.-i.; Park, S.; Park, J.; Park, S. Enhancing Electric Vehicle Charging Infrastructure: A Techno-Economic Analysis of Distributed Energy Resources and Local Grid Integration. Buildings 2024 , 14 , 2546. https://doi.org/10.3390/buildings14082546

Lee T, Yoon G, Kang B, Choi M-i, Park S, Park J, Park S. Enhancing Electric Vehicle Charging Infrastructure: A Techno-Economic Analysis of Distributed Energy Resources and Local Grid Integration. Buildings . 2024; 14(8):2546. https://doi.org/10.3390/buildings14082546

Lee, Tacklim, Guwon Yoon, Byeongkwan Kang, Myeong-in Choi, Sangmin Park, Junhyun Park, and Sehyun Park. 2024. "Enhancing Electric Vehicle Charging Infrastructure: A Techno-Economic Analysis of Distributed Energy Resources and Local Grid Integration" Buildings 14, no. 8: 2546. https://doi.org/10.3390/buildings14082546

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

MDPI

Subscribe to receive issue release notifications and newsletters from MDPI journals

Inventory Management Techniques of Heat Stores in Metro Manila: Basis for Improvement of Inventory Management System

Related Papers

Showing 1 through 3 of 0 Related Papers

IMAGES

  1. (PDF) Performance management systems: A comparative analysis

    research paper on performance management system

  2. (PDF) A STUDY OF PERFORMANCE MANAGEMENT SYSTEM AS A WAY TO PRODUCTIVITY

    research paper on performance management system

  3. (PDF) Factors Influencing Employee Perception on Performance Management

    research paper on performance management system

  4. Project Report On Performance Management System

    research paper on performance management system

  5. Performance Management Research Paper

    research paper on performance management system

  6. Performance Management System Essay Example

    research paper on performance management system

COMMENTS

  1. Effectiveness of Performance Management System for Employee Performance

    Performance management system effectiveness (PMSE) is the measure of alignment between employee and organizational objectives (Armstrong, 2015).Researchers (e.g., Kennerley & Neely, 2003; Kolich, 2009; Tan & Smyrnios, 2006) have substantiated that a careful implementation of an effective PMS ensures this consistency.An effective PMS implementation process necessitates that employees eagerly ...

  2. The Systematic Review of Effective Performance Management Systems in

    Abstract: This study aims to determine the impact of Performance Manage-. ment Systems (PMS) on employee and organizational p erform ance. The re-. searcher utilized explorator y research in the ...

  3. Performance Management: A Scoping Review of the Literature and an

    SUBMIT PAPER. Close Add email alerts. You are adding the following journal to your email alerts ... Linking Belgian employee performance management system characteristics with performance management system effectiveness: Exploring the mediating role of fairness. ... (2015). Embracing translational HRD research for evidence-based management: Let ...

  4. Performance Management: A Systematic Review of the Literature and an

    Performance Management (PM), in all its guises, is a core business practice across the majority of organizations; whether this occurs formally through an official organizational process or ...

  5. Full article: Performance management systems: reviewing the rise of

    Findings: The findings suggest there is a link between performance measurement systems and dynamics, but the link is weak, and the main research does not incorporate dynamic elements. There is an observable research community interest in managing information for informed decisions through digitalization.

  6. PDF Evaluating the Effectiveness of Performance Management

    We found only one article examin-ing this. Lawler (2003) evaluated the organization-level effective-ness of the PM systems of 55 Fortune 500 companies on two factors: effectiveness for influencing performance (the right kind of performance) and effectiveness for differentiating between top and poor performers/talent.

  7. How to Evaluate the Effectiveness of Performance Management Systems? An

    The purpose of this study is to propose an integrative model for evaluating the effectiveness of performance management system (PMS). This model aims to systematize the dimensions and criteria used in the literature and provide clarity in terms of evaluation possibilities. A comprehensive review of the literature was conducted to identify the dimensions, criteria, and causal relationships used ...

  8. (PDF) A Literature Review and Overview of Performance Management: A

    The paper covers the concept of performance management as a business analyst, scrum master, archeologist, and leader. The research delves into the founding history of performance management and ...

  9. "Measuring the Effectiveness of Performance Management Systems: A

    Review of Literature 1. Performance Management Systems Overview: Research suggests that effective performance management systems play a crucial role in aligning employee efforts with organizational goals (Aguinis, 2019). Various frameworks and methodologies have been employed to measure and enhance the effectiveness of these systems. 2.

  10. PDF EFFECTIVENESS OF PERFORMANCE MANAGEMENT SYSTEM ON ...

    below to address the knowledge gap in the discipline of performance management. 2.2 This paper is hinged research objectives 1. Explore the effectiveness of the Performance management system on employee performances. 2. Determine whether employees and managers understand the purpose of the performance management system 3.

  11. Performance management and employee engagement

    The first stage of the engagement management process involves performance agreement. The main activities at this stage involve goal setting and a review and agreement of the psychological contract. Employee engagement and improved performance. As shown in Fig. 1, employee engagement is expected to have a direct effect on improved job performance.

  12. PDF The Purposes of Performance Management Systems and Processes

    Purpose - This paper aims to organise, in a general typology, the different purposes of performance management systems and processes (PMSPs) that are discussed across the performance management literature in different functional areas. Design/methodology/approach - The typology is developed based on a traditional review of the performance

  13. Performance management system and its role for employee performance

    1. Introduction. Small and medium enterprises (SMEs) are essential to Ethiopia's economy and play an important role in creating jobs and reducing poverty [1].However, Ethiopian small and medium enterprises face some challenges to improve their employees' performance, including the lack of an appropriate performance management system [2].Employee morale and productivity might suffer as a result ...

  14. Performance management systems, innovative work behavior and the role

    Design/methodology/approach. Building on signaling theory and human resource management (HRM) system strength research, the authors designed a factorial survey experiment (n = 444) to examine whether PMSs stimulate IWB under different configurations of distinctiveness, consistency and consensus, as well as in the presence of transformational leadership.

  15. Effectiveness of Performance Management System

    cipal (employer). Hence, this paper fo-cuses on revisiting performance manage-ment system from agency theory perspec-tive and to study the process to enhance the effectiveness. Literature Review Performance management is achieved when each employee of an or-ganization is responsible and accountable for his/her own actions regardless of the

  16. Effectiveness of Performance Management System for Employee Performance

    Background. Performance management system effectiveness (PMSE) is the measure of alignment between employee and organiza-tional objectives (Armstrong, 2015). Researchers (e.g., Kennerley & Neely, 2003; Kolich, 2009; Tan & Smyrnios, 2006) have substantiated that a careful implementation of an effective PMS ensures this consistency.

  17. Performance Management System. A Literature Review

    3.3.1 Contingency Theory. Most of the literature on PMS design is grounded on the contingency approach. The contingency theory of management control system design has been developed for 30 years (Otley 1980); nonetheless it is still the main theoretical reference for a large part of the research in the field (Chenhall 2003).The main postulate of the theory relates to the fact that a range of ...

  18. PDF Performance management systems: A comparative analysis

    In this research a standard performance management system (Armstrong, 2006, 2003) is compared with the informal PMS as applied in LDO. The performance management system is shown in Figure 1. In this standard model there are five main components; Role definition, Performance agreement (Plan), Personal

  19. Putting the System Into Performance Management Systems: A Review and

    SUBMIT PAPER. Journal of Management. Impact Factor: 9.3 ... 2018. Putting the System Into Performance Management Systems: A Review and Agenda for Performance Management Research. Deidra J. Schleicher [email ... Algera J. A. 2004. Participation in the design of performance management systems: A quasi-experimental field study. Journal of ...

  20. PDF The Effects of Performance Management System on Employee ...

    Vol.3, No.4, 2014). Even the term "performance management" is much more recent, and so there is much less history to describe (DeNisi & Murphy, 2017,). This research has been focused on improving the performance of individuals, however, and the ultimate goal of performance management systems is to improve firm-level performance.

  21. Performance Management: Meaning, Stages, and Best Practices

    Performance management is the process of assessing and improving employee performance. It revolves around setting clear expectations, providing regular feedback, and supporting employee development. When done right, performance management ensures that team members understand what your company expects of them and how their roles contribute to ...

  22. If health organisations and staff engage in research, does healthcare

    One further trend in terms of the type of analysis is seen in the 11/95 papers that used bibliometric analysis as an indicator of the extent, and/or quality, of research engagement compared with some measure of the healthcare performance, in terms of processes and/or outcomes [30, 40, 50, 51, 56, 65, 76, 95,96,97, 100]; 10 of these papers are ...

  23. What Is a Performance Management System?

    Performance management systems are important because they encourage timely communication between employees and managers, which helps foster a culture of continuous feedback and accountability in the workplace. They also use data analytics to provide insights into employee performance management so HR professionals can make data-driven decisions.

  24. Performance Management System: A Strategic Tool for ...

    This paper deals with the effectiveness of PMS. The result of the study shows that a performance management system acts as a strategic tool and a powerful foundation for the employees to achieve ...

  25. References

    References provide the information necessary for readers to identify and retrieve each work cited in the text. Consistency in reference formatting allows readers to focus on the content of your reference list, discerning both the types of works you consulted and the important reference elements with ease.

  26. Identification of avoidable patients at triage in a Paediatric

    Crowding has been a longstanding issue in emergency departments. To address this, a fast-track system for avoidable patients is being implemented in the Paediatric Emergency Department where our study is conducted. Our goal is to develop an optimized Decision Support System that helps in directing patients to this fast track. We evaluated various Machine Learning models, focusing on a balance ...

  27. Buildings

    The electric vehicle (EV) industry has emerged in response to the necessity of reducing greenhouse gas emissions and combating climate change. However, as the number of EVs increases, EV charging networks are confronted with considerable obstacles pertaining to accessibility, charging time, and the equilibrium between electricity demand and supply. In this paper, we present a techno-economic ...

  28. Inventory Management Techniques of Heat Stores in Metro Manila: Basis

    stores, presenting a comparative analysis of Barcode and Manual Inventory Management systems. The primary objective is to glean insights that can enhance current inventory practices, considering the perspectives of employees experienced in inventory management. The study's methodology involved meticulously creating survey questions, drawing on pertinent literature to ensure precision and ...

  29. (PDF) Performance management systems: A comparative analysis

    Performance management systems: A com parative. analysis. Jawaria Andleeb Qureshi, Asad Shahjehan*, Zia-ur-Rehman and Bilal Afsar. Department of Management Sciences, Ha zara University, Manshera ...

  30. Assenagon Asset Management S.A. Sells 54,586 Shares of Veeva Systems

    Veeva Systems Stock Performance. Shares of NYSE:VEEV traded up $1.56 during trading hours on Monday, hitting $194.06. 651,901 shares of the company's stock traded hands, compared to its average volume of 960,788. Veeva Systems Inc. has a 52 week low of $162.72 and a 52 week high of $236.90. The company's fifty day simple moving average is $185.73 and its two-hundred day simple moving average ...