governance a systematic literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Health Policy Plan
• v.32(5); 2017 Jun

Frameworks to assess health systems governance: a systematic review

Thidar pyone.

Centre for Maternal and Newborn Health, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA

Helen Smith

Nynke van den broek, associated data.

Governance of the health system is a relatively new concept and there are gaps in understanding what health system governance is and how it could be assessed. We conducted a systematic review of the literature to describe the concept of governance and the theories underpinning as applied to health systems; and to identify which frameworks are available and have been applied to assess health systems governance. Frameworks were reviewed to understand how the principles of governance might be operationalized at different levels of a health system. Electronic databases and web portals of international institutions concerned with governance were searched for publications in English for the period January 1994 to February 2016. Sixteen frameworks developed to assess governance in the health system were identified and are described. Of these, six frameworks were developed based on theories from new institutional economics; three are primarily informed by political science and public management disciplines; three arise from the development literature and four use multidisciplinary approaches. Only five of the identified frameworks have been applied. These used the principal–agent theory, theory of common pool resources, North’s institutional analysis and the cybernetics theory. Governance is a practice, dependent on arrangements set at political or national level, but which needs to be operationalized by individuals at lower levels in the health system; multi-level frameworks acknowledge this. Three frameworks were used to assess governance at all levels of the health system. Health system governance is complex and difficult to assess; the concept of governance originates from different disciplines and is multidimensional. There is a need to validate and apply existing frameworks and share lessons learnt regarding which frameworks work well in which settings. A comprehensive assessment of governance could enable policy makers to prioritize solutions for problems identified as well as replicate and scale-up examples of good practice.

Key Messages

• Health system governance is one of the neglected agendas in health system research.
• There is currently a lack of evidence with regard to how governance can and is assessed at both national and sub-national level.
• Existing frameworks can be adapted to assess governance overall or specific components of governance.

Introduction

Governance is defined as the rules (both formal and informal) for collective action and decision making in a system with diverse players and organizations while no formal control mechanism can dictate the relationship among those players and organizations ( Chhotray and Stoker 2009 ). Some authors criticize the concept of governance for being too vague ( Schneider 2004 :25) and there is confusion over how best to conceptualize it ( Kohler-Koch and Rittberger 2006 :28). Governance has been discussed in many disciplines such as political science, economics, social science, development studies and international relations using different theories. Governance matters as it is concerned with how different actors in the world function and operate and the reasons behind their decisions.

Political scientists are of the opinion that governance is not a science which can be ‘adequately captured by laws, statues or formal constitutions’ ( Chhotray and Stoker 2009 ). Governance is not easily attained with laws, statutes or formal constitutions, rather it is a system level concept (macro level) in which systems or societies are driven by networks. Each network involves multiple nodes (organizations) with many linkages collaborating on different activities ( McGuire 2010 :437). The assumption is that passing a law or decree from a formal authority cannot in itself achieve engagement of key actors, and negotiation is key to success of governance within networks ( Chhotray and Stoker 2009 ). Political scientists have also expressed concerns that there are insufficient tools to hold people accountable as governance is characterized by complicated policy networks and responsibility is diffused and shared among many stakeholders ( Stoker 2006 ).

Governance in new institutional economics focuses on the role of institutions which shape interactions among actors within the constraints of the institutions ( Chhotray and Stoker 2009 ). Choices are made within the context of institutional rules that shape and govern what is decided ( Chhotray and Stoker 2009 ). This concept of governance has received support from other disciplines including political science. New institutional economists describe governance as a series of actions which secure voluntary co-operation among key actors.

Governance is becoming more important in international development, particularly due to the movement towards ‘good governance’ in international aid. The World Bank has played a central role in bringing governance into the development agenda, introducing the concept of ‘good governance’ in 1989 in a landmark report on sustainable growth in sub-Saharan Africa ( World Bank 1989 ). The report encouraged donor countries to be ‘selective’ and to give aid to countries with a ‘good policy environment’ ( Chhotray and Stoker 2009 ). In many ways, governance has been used as a political tool in international development, although this is often denied ( Chhotray and Stoker 2009 ).

In relation to health, governance was introduced in the World Health Report in 2000, where the World Health Organization (WHO) defined it in terms of ‘stewardship’, and called for strategic policy frameworks combined with effective oversight, regulation, incentives and accountability. This definition is based on political ideology; that the health system can be influenced by transparent rules, governed by effective oversight and strong accountability ( WHO 2007 ). More recently, health system governance has been described as ‘an aggregation of normative values such as equity and transparency within the political system in which a health system functions’ ( Balabanova et al. 2013 ). As efforts to strengthen health systems and health service delivery have accelerated during the last few decades, governance has received increasing attention. Prominent international development partners have described governance as being the ‘most important factor’ for poverty alleviation and development ( Graham et al . 2003 ).

Governance comprises different functions both within and outside the health sector. In the literature these are commonly described as ‘principles’, ‘concepts’, ‘dimensions’, ‘components’ or ‘attributes’. These terms tend to be used synonymously in the literature. For this review, we used the term ‘principles’. Research is needed both to explore each of the principles of governance in more depth and, to describe and assess governance more generally, in order to identify ways of improving health systems ( Lowenson 2008 ).

Our own work is predominantly around improving availability and quality of maternal and newborn health services in low- and middle-income countries; and we hypothesize that governance principles, if implemented effectively, can make a difference to the functioning of healthcare facilities. Our primary aim was to understand which frameworks for assessing governance in health systems have been developed and how these try to operationalize and/or assess how governance principles at different levels of a health system are implemented. Duran and Saltman (2015) describe hospital governance as dependent on three interrelated levels; (1) the macro-level (health system within which the health facility operates); (2) the meso-level (institutional decision-making) and (3) the micro-level (hospital management focusing on day-to-day operations). Our motivation for summarizing and critiquing frameworks for governance is to understand whether and how they might inform the assessment of governance at the operational service delivery level of a health system (the health facility). In doing so, we acknowledge that frameworks can provide direction on what to consider in assessing governance, but, given the diffuse nature of governance, there is unlikely to be a generic way of assessing governance in health systems.

We conducted a systematic review of the literature to: (1) describe and critique how the concept of governance and the theories underpinning it have been applied to health systems globally; and (2) identify if and how frameworks have been developed and used to assess governance in the health system.

Search strategy and inclusion criteria

We developed two inclusion criteria to meet the above mentioned review objectives. For the first objective, we included any type of report or peer reviewed journal article that reported frameworks for assessing or defining health systems governance. For the second objective we were interested only in articles reporting research or evaluations of the application of governance frameworks ( Table 1 ). We were only interested in articles reporting on governance frameworks which can be applied to the health sector, irrespective of disciplines. The search was limited to English language articles between January 1994 (the year when the term Governance was introduced by the World Bank) and February 2016.

Inclusion criteria used to select papers for each stated objective

We searched five electronic databases (Scopus, Medline, CINAHL, Global Health Database, Cochrane Library) using key words combined with the Boolean operators (AND, OR). For example, the key words for governance (governance, leadership, accountability, stewardship) were combined with terms relating to the health system (healthcare system, healthcare industry, healthcare reform, health system strengthening) and terms for frameworks (model, framework, indicator, definition, measure). All the terms were searched in abstracts, key words, subject headings, titles and text words. We searched Medline first, and adapted this search strategy for use with other databases. Search strategies used in each database, including search terms, search strings and results, are outlined in Supplementary Table S1 .

In addition to the database search, we searched the online archives of specific journals that publish research on health systems and policy including ‘ Health Policy and Planning ’ and ‘ Health Policy ’ using ‘health systems governance’ as the key search term. Web portals of institutions including the Basel Institute for Governance, the World Bank and USAID Leadership, Management and Governance project were also searched. Furthermore, we checked the reference lists of studies that met our inclusion criteria and contacted the authors of identified frameworks to ask for any unpublished reports which were considered relevant.

Assessment of quality of included studies

We did not appraise the quality of studies describing frameworks health systems governance, as these were largely descriptive reports (Objective 1). For objective two, we included articles reporting empirical research, and we assessed the quality of these studies using simple criteria based on published checklists ( Crombie 1996 ). Because the study designs were diverse, we appraised studies based on: the description of the study (aim, participants, methods, outcomes); the methods (appropriate to the aim, selection of participants, valid and reliable data collection methods, and adequate description of analysis) and presentation of the study findings. For qualitative studies, this included questions about appropriateness and reliability of analysis; and for those reporting quantitative data, we assessed whether the basic data were adequately described, and whether statistical significance was assessed.

The review identified a total of 10 empirical studies of which 9 were peer-reviewed, 3 were rated as high and 7 as medium quality. ( Supplementary Table S2 ) All studies provided adequate descriptions regarding information of the study such as aims, study participants, methods employed and their intended measures. Seven studies used qualitative methods (interviews, focus group discussions), one used a quantitative method (survey) ( Abimbola et al. 2015b ) and two were mixed-methods studies (Mutale et al . 2012; Avelino et al. 2013 ). Seven studies provided information on how study participants were selected ( Huss et al. 2011 ; Avelino et al. 2013 ; Mutale et al. 2013 ; Vian and Bicknell 2013 ; Abimbola et al. 2015a , b , 2016).

Seven studies provided information on methods of data analysis Baez-Camargo and Kamujuni 2011 ; Avelino et al. 2013 ; Mutale et al. 2013 ; Vian and Bicknell 2013 ; Abimbola et al. 2015a , b ,2016. Among the seven studies which used qualitative methods, quotes were included in five; ( Baez-Camargo and Kamujuni 2011 ; Huss et al. 2011 ; Vian and Bicknell, 2013 ; Abimbola et al. 2015a , 2016). All three studies which conducted statistical analysis provided a rationale for statistical calculations used.

Synthesis of review findings

As governance originates from many different disciplines, we undertook an in-depth analysis offering a theory-informed critique of the identified frameworks and of the literature on governance, extending beyond health systems. The findings of included studies were synthesized using narrative synthesis which is useful in synthesizing different types of studies without losing the diversity in study designs and contexts ( Lucas et al. 2007 ; Barnett-Page and Thomas 2009 ; Wong et al. 2013 ). Included studies are summarized by objective in the results section, and by grouping them by the disciplines from which the frameworks originate.

Description of included studies

We identified a total of 373 articles through database searching and 39 through other sources, of which 25 met the inclusion criteria ( Figure 1 ) ( Table 2 ).

Flow diagram of study selection procedure and results (adapted from PRISMA 2009)

Overview of governance frameworks for health systems by type of discipline used to develop the framework

Sixteen articles describe frameworks for assessing governance and 10 empirical research studies which describe how frameworks can potentially be used to assess health systems governance were identified.

One previous review on governance (a non-peer reviewed report) was conducted to inform the development of a framework which would be specifically used in surveys of the countries included in the Health Systems 20/20 project ( Shukla and Johnson Lassner 2012 ). The report provides an overview of the current literature on governance in the health sector. The authors discuss 10 principles termed ‘enablers’ in detail and outline existing frameworks; highlighting how effective governance is associated with health outcomes in three country-level studies.

I. Description and critique of governance frameworks

We identified a total of 16 frameworks developed to assess governance in the health system. Of these, six frameworks were developed based on theories from new institutional economics; three are primarily informed by political science and public management disciplines; three arise from the development literature and four use multidisciplinary approaches ( Table 3 ).

Summary table of governance frameworks identified, grouped by discipline

Frameworks originating from new institutional economics

Six frameworks conceptually originate from New Institutional Economics: EC (2009), Baez-Camargo (2011 ), Brinkerhoff and Bossert (2008 ), Baez-Camargo and Jacobs (2013 ), Cleary et al. (2013 ) and Abimbola et al. (2014 ). Among these, five use ‘principal–agent’ theory ( Brinkerhoff and Bossert 2008 ; European Commission 2009; Baez-Camargo 2011 ; Baez-Camargo and Jacobs 2013 ; Cleary et al. 2013 ) while Abimbola et al. (2014) use Ostrom’s theory of ‘common pool resources’.

Principal–agent theory

In ‘principal–agent’ theory, a ‘principal’ hires or contracts an ‘agent’ to undertake a particular service ( Chhotray and Stoker 2009 ). Agents may have similar as well as different objectives from those of the principal. Agents, usually have more information than the principal, providing them with an advantage to pursue their own interests at the expense of the principal. Fundamentally, the theory looks at how much of the value that the agent produces should go back to him/her in the form of incentives i.e. the agent (healthcare provider) produces certain services for the principal (the government), for which the agent expects some form of payment ( Chhotray and Stoker 2009 ).

The other distinctive feature of the ‘principal–agent’ theory is that the principal does not have complete control over the agent and only has partial information pertaining to the behaviour (production) of the agent ( Stoker 1998 ). This can lead to difficulties such as selection of agents, negotiation of services and monitoring of the information. Therefore, governance frameworks using the ‘principal–agent’ theory take into account the uncertainty and complexity of the outcomes of the behaviour of the agent ( Stoker 1998 ).

Frameworks to assess health systems governance that draw on ‘principal agent’ theory, assume that governance is the result of interactions among principals and agents with diverse interests. Two key assumptions using ‘principal–agent’ theory are; (1) there are incentives and sanctions for the different actors which are performance-based and are used to stimulate accountability and, (2) information asymmetry and power difference among different groups. Healthcare users are normally regarded as ‘principals’ while the state and healthcare providers are ‘agents’ providing healthcare services to users ( Brinkerhoff and Bossert 2008 ; European Commission 2009; Baez-Camargo 2011 ; Baez-Camargo and Jacobs 2013 ; Brinkerhoff and Bossert 2013 ; Cleary et al. 2013 ). Agents provide services to principals as long as they have some incentive to do so, but they have more information than principals. At the same time, principals will find ways to overcome the information asymmetry without incurring high transaction costs. For instance, users will look for alternative providers by comparing price, quality or value. In addition, context matters in these frameworks as the ‘principal–agent’ model is a highly complex set of interactions and not a closed system. It helps to explore how policy makers respond to citizen demands, how health service providers and users engage to improve service quality, and how service providers and users advocate and report on health outcomes.

The framework by Brinkerhoff and Bossert (2008 , 2013 ) is based on a World Bank (2004) accountability framework. The framework depicts three principal–agent relationships: government and healthcare providers; healthcare providers and citizens; and government and citizens. The other framework which uses the ‘principal–agent’ theory is the governance framework of the European Commission (2009). The EC (2009) framework aims to assess governance at sector level especially in the context of development and aid assistance worldwide. The EC framework takes into account the importance of context and assessment starts with context analysis and stakeholder mapping. Similarly to the framework by Brinkerhoff and Bossert (2008 , 2013 ), the EC framework considers power, interactions and functions of stakeholders as core governance issues, but also includes principles of participation, inclusion, transparency and accountability. Among different principles, the framework focuses on accountability among different stakeholder groups. Though the framework is intended to be used for development and aid assistance, the framework does not include citizens among the defined clusters of stakeholders. The EC (2009) framework has a ready-to-use tool with detailed instructions. Examples from previous EC projects globally are provided with suggestions on how to improve governance. Although the authors do not empirically test the framework, they suggest how it might be applied it to a fictional country in sub-Saharan Africa.

Baez-Camargo (2011) and Baez-Camargo and Jacobs (2013) proposed an analytical framework of ‘social-accountability’ by adapting the World Bank accountability model ( World Bank 2004 ). Using the ‘principal–agent’ theory, Baez-Camargo (2011) presented incentives and sanctions within two routes towards accountability: short (direct) and long (indirect) routes. Direct accountability is most suitable in the competitive market where citizens can ‘voice’ their preference or choose other alternatives (exit). On the other hand, with indirect accountability, the link between citizens and healthcare providers is considered ‘indirect’ as the government agent is involved in the accountability relationship; citizens hold the government agent accountable either through political representation (votes) and the government holds healthcare providers accountable to deliver healthcare services. Direct accountability has received the most attention as it can be promoted either through citizens’ participation in service planning, or voicing concern about service providers’ performance (voice), or through citizens’ choosing other providers (exit). However, it is important to be careful about applying the concept of direct accountability to health care in settings where market competition fails to provide healthcare services to the most vulnerable groups. The authors include tools for key informant interviews.

Another framework using ‘principal–agent’ theory is the accountability assessment framework for low- and middle-income countries developed by Cleary et al. (2013) . By adapting the Brinkerhoff and Bossert (2008) framework, the authors emphasize the accountability pathways among three groups of key actors (politicians/policy makers; healthcare providers and citizens). The Cleary framework claims to assess both external and internal accountability mechanisms via three critical factors: resources, attitudes and values. The authors highlight that adequate resources are critical for the health system to function properly while it is important to understand the attitudes of healthcare providers and policy makers without neglecting the values of citizens.

Theory of common pool resources

Our review identified one framework which uses theory derived primarily from economics; Elinor Ostrom’s theory of ‘common pool resources’ ( Ostrom 1990 ). This theory describes governance as an autonomous system with self-governing networks (or systems) of actors ( Stoker 1998 ). The theory assumes that actors in self-governing networks can not only influence government policy but can also take over some of the business of the government ( Stoker 1998 ). Ostrom’s theory focuses on creating different institutional arrangements to manage open resources which are finite. Communities can form self-organized networks or systems composed of interested actors who will develop incentives and sanctions to manage the resources on their own ( Stoker 1998 ). The theory assumes that self-organized systems are more effective than regulation imposed by the government as there will be increased availability of information and reduced transaction costs ( Stoker 1998 ). Indeed, the theory postulates that in situations where government is ‘under-governed’, social norms fill those gaps (Olivier de Sardan 2015). A similar assumption is highlighted by Dixit (2009) civil-society organizations and non-governmental organizations emerge to fill gaps in functioning when government organizations serve poorly. The theory proposes that there are three levels of a common pool resource problem: (1) an operational level where the working rules are set, (2) a collective level where communities set their own rules, and, (3) a constitutional level from where the set rules originate ( Ostrom, 1990 :45).

Using Ostrom’s theory of ‘common pool resources’, Abimbola et al. (2014) developed a multi-level framework to analyse primary healthcare (PHC) governance in low- and middle-income countries. The authors borrowed the concept of ‘governing without government’ in situations where overall governance situations are not functioning. In such situations, communities with similar interest might develop their own rules and arrangements to manage the common pool. Ostrom argued that self-governing arrangements lead individuals or groups to cope with problems by constantly going back and forth across levels as their key strategy. Abimbola’s framework (2014) describes the three collective levels of health system hierarchy as; (1) operational (citizens and healthcare providers), (2) collective (community groups) and (3) constitutional governances (governments at different levels). A multi-level framework is believed to be more effective at assessing governance than a single unit assessment. Operational and collective governance can mitigate the failure of constitutional governance, although, there is also some overlapping of roles and responsibilities.

Frameworks originating from political science and public administration

Three frameworks conceptually originate from political science and public administration disciplines: Berlan and Shiffman (2012) , Brinkerhoff (2004) and Brinkerhoff and Goldsmith (2004) . None of the frameworks mention any particular theory on which their frameworks are based. The concept of governance for political scientists focuses on ‘formal institutions, accountability, trust and legitimacy’ for governance ( Pierre and Peters 2005 :5). They are interested to see how collective decisions are made among key actors (both government and non-government actors) with different power ( Chhotray and Stoker 2009 ). Thus, governance from political science and public administration focuses on both inputs (the processes) and outputs (results of governing networks) ( Chhotray and Stoker 2009 ).

Berlan and Shiffman’s framework (2011) assumes that healthcare providers in low- and middle-income countries have limited accountability to their consumers as a result of both health system and social factors. Oversight mechanisms, revenue source and nature of competition are related to the health system while consumer power and provider norms are considered under social factors. Their framework helps to identify factors which shape the accountability of healthcare providers. In addition, social interactions and norms operating within the system and context are prominent features of this framework.

Brinkerhoff’s framework (2004) is also based on accountability, and aims to map out public accountability mechanisms: financial, performance and political accountability. In this framework, performance accountability is defined as agreed upon targets which should theoretically be responsive to the needs of the citizens. Political accountability emphasizes that electoral promises made by the government should be fulfilled. Brinkerhoff highlights the need to map out the accountability linkages among key actors and to examine actors’ interactions as too few linkages can lead to corruption while too many can undermine accountability effectiveness. Together with his framework, Brinkerhoff proposes three strategies to strengthen accountability; (1) addressing fraud, misuse of resources and corruption, (2) assuring compliance with procedures and standards and (3) improving performance. The framework includes an accountability assessment matrix which allows the user to rate accountability linkages among key actors.

The third framework that draws on political science assesses the patron–client relationship or clientelism in health systems ( Brinkerhoff and Goldsmith 2004 ). Despite the unpopularity of clientelism, it is regarded as an essential principle of governance which can affect corruption and accountability mostly at macro/national level. The purpose of the framework is to identify reasons why clientelistic practices persist and the authors use the concept of realist evaluation theory comprising of context, actions (mechanisms) and outcomes. Although the framework has not been used in the field, the authors present a diagnostic framework with sample questionnaires.

Frameworks originating from international development

In the development literature, governance focuses on predefined principles which development specialists believe to be critical for ‘good governance’ in aid assistance. The three frameworks identified (Islam et al . 2007; Kirigia and Kirigia 2011 ; Mikkelsen-Lopez et al. 2011 ) focus primarily on how governance is defined, how it can facilitate effective aid policy, and, unlike any of the other frameworks, those in international development are concerned with how governance might be measured. Kauffman and Kraay (2007) propose to measure governance in two ways using rule-based measures (e.g. a policy or a procedure exists) and outcome-based measures (e.g. the policy has been implemented or the rule has been enforced) ( Chhotray and Stoker 2009 ).

Islam (2007) present a health systems assessment manual which includes a framework to assess governance, developed under the Health Systems 20/20 project (USAID). The aim is to guide data collection providing a rapid but comprehensive assessment of key health system functions. Based on the six domains of the health system (1) service delivery; (2) health workforce; (3) health information systems; (4) access to essential medicines; (5) financing; and (6) leadership and governance. This framework groups indicators into general governance (e.g. voice and accountability; political stability; government effectiveness; rule of law; regulatory quality and control of corruption) and health system specific governance indicators (e.g. information/assessment capacity; policy formulation and planning; social participation and system responsiveness; accountability; and regulation). The authors suggest various sources of data for the different indicators, including interviews with relevant key stakeholders and desk-based review of relevant documents and reports.

Another framework that attempts to measure governance is one based upon Siddiqi et al. (2009) , which also includes principles of macroeconomic and political stability ( Kirigia and Kirigia 2011 ). The authors emphasize that development in health cannot occur without political and economic stability in the form of a national economic development plan or poverty reduction strategy, a medium-term government expenditure framework, and a non-violent electoral process. The authors argue that individual and aggregate scores of governance are needed to alert policy makers to areas needing improvement. This is the only framework identified in our review which tries to quantify governance by using rule-based measures such as existence of certain policy or guidelines. The authors propose a scoring system that determines whether governance is very poor (0%) or excellent (100%) for each function. Kirigia and Kirigia (2011) argue that scoring allows assessors to identify areas for improvement, and an overall index representing the overall governance situation in any given country can be calculated.

The final framework ( Mikkelsen-Lopez et al. 2011 ) is based on systems thinking, and uses a problem-driven approach to assess governance in relation to an identified problem to highlight the barriers to good governance. The framework assesses governance in all four levels of a health system (national, district, facility and community) using the established WHO health system building blocks and five proposed principles of governance: (1) strategic vision and policy design; (2) participation and consensus orientation; (3) accountability; (4) transparency; and (5) control of corruption. The authors developed this approach in response to other frameworks on governance that provide snapshots of any given governance situation, but are unable to identify specific areas of weakness and/or how to intervene. However, despite providing a way to identify barriers to good governance, the framework does not easily allow for comparisons between different contexts, and it is not clear if it has actually been applied in practice.

Frameworks originating from more than one discipline

Four frameworks appear to be based on principles of more than one discipline ( Vian 2008 ; Siddiqi et al. 2009 ; Baez Camargo and Jacobs 2011; Smith et al. 2012 ). Three of these ( Vian 2008 ; Siddiqi et al. 2009 ; Baez Camargo and Jacobs 2011) draw on the ‘institutional analysis’ theory of North (1990) , originally derived from new institutional economics. The frameworks also seem to reflect predefined governance principles in line with the international development literature.

Theory of ‘institutional analysis’

Douglas North’s theory of institutional analysis assumes that markets are created and maintained by institutions. North defined ‘institutions’ as the rules of the game and ‘organizations’ as the players. Institutions consist of formal rules and informal constraints while organizations consist of groups of individuals with common objectives ( North 1990 ). North’s principal argument is that individuals within an institution have certain opportunities which are the result of specific formal and informal constraints that constitute the institutions. Using the theory of North (1990) , Vian (2008 ), Siddiqi et al. (2009) and Baez Camargo and Jacobs (2011) highlighted that institutional analysis is key to assessing governance in order to understand the institutional arrangement and rules set by the organizations. A mapping of the power distribution can be used to identify the key decision makers who affect the behaviour of health system actors.

In addition to application of North’s theory of institution analysis, Siddiqi et al. (2009) propose a comprehensive framework to assess governance based on the UNDP principles of governance. This framework includes ten principles, disaggregated into 63 broad questions under three relevant domains: context, processes and outcomes. In conceptualizing governance in this way, the authors suggest that their framework could be used to compare governance functions across countries. The framework is intended for use at both national (policy formulation) and sub-national levels (policy implementation and health facility levels) to assess all essential principles of health systems governance; something which other frameworks do not aim to do. In particular, the potential for application of the framework at subnational level is a unique feature as most other governance frameworks are developed for macro-level assessment.

Baez-Camargo and Jacobs (2011) propose an ‘inputs, processes and outputs’ framework for health systems governance in low-income countries. The authors acknowledge the existence of other frameworks to assess health systems, but set theirs apart by focusing on generating information on the complex context within which the health system operates. The framework draws on the values of good governance articulated in the development literature, and ‘Institutional analysis’ to map out key stakeholders and the power distribution among them. The framework is presented as a visual process map of causal links between inputs, processes and outcomes, which they believe helps to provide a better explanation of governance and easier application of the framework. The authors provide detailed methodology, tools and procedures for using the framework in practice, but acknowledge that their model cannot assess health systems governance in its entirety. It is recommended for use in contexts where a particular problem has first been identified.

Vian’s (2008) framework specifically analyses corruption in the health system from the perspective of the government. It draws on North’s principal argument that key players in the health system have certain opportunities which are the product of formal and informal rules and constraints set by institutions ( North 1990 ). The author also employs ‘principal–agent’ theory as the framework takes into account asymmetric information among different actors with diverse interests within a health system. The framework is based on the assumption that corruption in the health sector is driven by pressures of government agents to abuse, opportunity to abuse, and social factors supporting abuse of the system. Therefore, the framework is diagnostic in nature as it aims to identify potential abuse that can occur at each step of a health service delivery process.

Smith et al. (2012) describe a ‘cybernetic’ framework for leadership and governance which uses systems theory. This theory is interdisciplinary and is concerned with discovering patterns in the way systems (including health systems) operate. Smith et al. consider it important to view governance as hierarchical (rules and responsibilities for allocating resources) and horizontal (both incentives and the market regulate purchasing power, and systems produce common values and knowledge through professional norms). Cybernetics focuses on how systems use information, and how systems monitor actions to steer towards their goals. The framework includes three key principles related to this: setting priorities, accountability (inputs into the health system) and performance monitoring (output). The framework focuses on the leadership principle of governance and was developed for use in health systems in high-income countries, so would require adaptation to low-and middle-income settings.

II. Description of how frameworks have been applied to assess governance in health systems

Among the 16 frameworks identified that can potentially be used to evaluate health systems governance, only 5 ( Brinkerhoff and Bossert 2008 ; Siddiqi et al. 2009 ; Baez-Camargo and Jacobs 2011 ; Smith et al. 2013; Abimbola et al. 2014 ) have actually been applied. ( Supplementary Table S2 ).

Among the 12 publications describing how frameworks have been applied, seven use ‘principal–agent’ theory; two make use of the theory of ‘common pool resources’; two use North’s institutional analysis; and one uses ‘cybernetics’ theory.

Studies which used ‘principal–agent’theory

Among frameworks using ‘principal–agent theory’, Brinkerhoff and Bossert’s framework is the most commonly applied (five studies; Mutale et al. 2012; Vian et al. 2012 ; Brinkerhoff and Bossert 2013 ; Cleary et al. 2013 ; Ramesh et al. 2013 ) while the other three studies ( Huss et al. 2011 , Avelino et al. 2013 ; Vian and Bicknell 2013 ) used a variant of the ‘principal–agent’ theory. The USAID health system assessment team used Brinkerhoff and Bossert’s governance framework in their manual for assessing health systems. According to Health Systems 20/20, the manual is currently used in 23 Health Systems 20/20 projects funded by the USAID in countries in East, West, and Southern Africa, as well as in the Caribbean islands ( Health Systems 20/20, 2012 ).

Mutale et al. (2012) adapted Brinkerhoff and Bossert’s framework to assess governance at health facility level in Zambia while Ramesh et al. (2013) used the framework at national level in China. Cleary et al. (2014) adapted Brinkerhoff and Bossert’s framework to assess accountability mechanisms in low- and middle-income countries. Vian et al. (2012) employed Brinkerhoff and Bossert’s framework to assess corruption in the Vietnamese health system.

Three other studies ( Huss et al. 2011 ; Avelino et al. 2013 ; Vian and Bicknell 2013 ) applied the ‘principal–agent’ theory to assess governance in Brazil, India and Lesotho. Huss et al. (2011) applied a variant of the ‘principal–agent’ model in their assessment of governance focusing on corruption in Karnataka State, India. Contrary to the traditional application of ‘principal–agent’, Huss et al. refer to the ‘state’ as ‘principal’ while ‘public service providers’ are ‘agents’ to deliver certain services for ‘citizens’.

All studies used two principal–agent relationships—the relationship between citizens and government and between government and healthcare providers—with the exception of Vian and Bicknell (2013) who use a single principal–agent model (state-healthcare provider). The studies evaluate the principal and agent engage and interact to accomplish a collective effort and clearly highlight the importance of information asymmetry.

Studies which used ‘multilevel framework’ of governance

Two studies ( Abimbola et al. 2015a , b ) applied the ‘multilevel framework’ by Abimbola et al. (2014) .

Abimbola et al. (2015a) adapted the ‘multilevel framework’ to identify the effect of decentralization on retention of PHC workers in Nigeria. The framework was used to assess government, communities and intrinsic health workers’ factors influencing retention of PHC workers in a decentralized health system. The framework helped identify incentives for, and motivation of, PHC workers and the reasons they remain in post despite socio-economic hardship.

The ‘multilevel framework’ was also applied to provide recommendations to improve health system governance at operational level among tuberculosis (TBC) patients in Nigeria ( Abimbola et al. 2015b ). The framework was used to assess the three different levels of governance: constitutional (federal government); collective (communities) and operational (healthcare providers at local health market). In this, the concept of Williamson’s Transaction Cost Theory (1979) was used to identify the costs incurred by TBC patients to receive appropriate anti-TBC treatment from a qualified provider working within the health system. Transaction costs are difficult to measure thus Williamson suggested looking into ‘the issues of governance comparatively’. The central argument of Williamson’s theory is that ‘high transaction costs’ can be attributed to governance failure which requires looking for alternative modes of governance to achieve ‘economizing’ results ( Williamson 1999 ). In both studies, self-governing individuals at three levels of a system are trying to overcome a common problem by identifying ways which are workable for them.

Studies which used North’s theory of ‘institutional analysis’

Siddiqi et al. (2009) used their own framework to assess governance of the health system in Pakistan, and explored governance principles in depth using qualitative interviews. The authors assessed three different levels of the health system—national (policy formulation) and sub-national levels (policy implementation and health facility levels). The authors highlighted the importance of understanding the socio-political context of a country and show that the principles of health systems governance are value driven. In addition, Siddiqi et al. emphasize that health system governance can be improved without improving the overall governance of a country.

Baez-Camargo and Kamujuni (2011) conducted an assessment of the governance of the public sector drug management system in Uganda using the framework of Baez-Camargo and Jacobs (2011) . The assessment started with an institutional mapping which included interviews with both formal and informal sectors of the supply chain in Uganda. Focus group discussions were also conducted with healthcare providers, patients and representatives of patient advocacy groups.

Smith et al. (2012) applied their cybernetic framework at national level to seven health systems in high-income settings (Australia, England, Germany, the Netherlands, Norway, Sweden and Switzerland). The framework is composed of three key nodes of governance (setting priorities, accountability and performance monitoring) which serve as the guiding principle for assessing hierarchy, market and network governance. One important lesson highlighted by the authors is that competency and capacity at the different levels of a health system are crucial for successful implementation of the leadership and governance model.

This systematic review brings together the literature on health systems governance, firstly by describing and critiquing how the concept of governance and the theories underpinning it have been applied to health systems, and secondly by identifying which frameworks have been used to assess health systems governance, and how this has been done to date globally. A total of 16 frameworks were identified, which, in principle, can be used at national (policy formulation) and sub-national (policy implementation) levels of a health system. Frameworks originate mainly from three disciplines: (1) new institutional economics; (2) political science and public administration; and (3) the international development literature.

The most commonly used theories which underpin the available frameworks originate from new institutional economics and include the ‘principal–agent’ theory, Douglas North’s theory of institutional analysis and Elinor Ostrom’s theory of ‘common pool resources’. Frameworks that originate from the development literature tend to pre-define principles of governance and are the only ones to attempt to measure governance (for instance, Kirigia and Kirigia 2011 ). The majority of frameworks assess overall governance while some assess specific principles of governance such as accountability, corruption and patron–client relationship.

Most frameworks assess governance in health systems using qualitative methodology, based on the premise that governance is the result of interactions among different actors within a health system, and that studying the reasons for and the extent of interaction can be used to document good governance. Other authors propose using mixed methods; collecting data on framework indicators (e.g. Mutale et al. 2012) in combination with in-depth exploration of specific problems identified.

It is encouraging to see that there are three frameworks that have been used to assess governance at all levels of the health system; Brinkerhoff and Bossert (2008 ), Siddiqi et al. (2009 ) and Abimbola et al. (2014 ). Governance is a practice, dependent on arrangements set at political or national level, but which needs to be operationalized by individuals at lower levels in the health system; multi-level frameworks acknowledge this and recognize the importance of actors at different levels. Some assessment frameworks explicitly mention pre requisites needed for successful application, such as the framework by Baez-Camargo and Jacobs (2011) which requires a governance problem to be already identified, and the cybernetic model presented by Smith et al. (2013) which requires users’ familiarity with the health system.

This review also illustrates that health system governance is complex and difficult to assess; the concept of governance originates from different disciplines and is multidimensional. Governance more generally has been debated and studied from many different perspectives. This review attempts to synthesize how these perspectives have led to the development of governance in health systems. Critical analysis shows that frameworks for assessing governance may be applicable in one setting but not another. There is no single, agreed framework that can serve all purposes as the concept of governance will likely continue to be interpreted openly and flexibly. However, for governance principles to contribute to health system strengthening in countries, and ultimately to impact on outcomes, it is critical to at least evaluate and monitor if and how governance works (or not) in practice. As each health system operates in its own context, and different components of governance may need to be prioritized over others in different settings and at different times, it is important that any assessment of governance recognizes the particular circumstances and has a clear purpose. Assessing health systems governance can raise awareness of its importance to health policy makers, identify problems or conversely, document success stories. This can encourage and catalyse improvement in health systems. The aim of this review was to provide an overview of frameworks available and describe how they have been developed, adapted or applied to assess health systems governance in operation. We recognize that the main utility of the synthesis is not to identify features of a single agreed framework, rather the frameworks identified and reviewed can help assessors to identify relevant questions to ask of health systems governance, and identify elements that could be included in an assessment.

Outside of the limited evidence on how governance can be assessed in health systems, this review also highlights examples of how governance has been assessed in other disciplines. Both rules-based and outcomes-based approaches to assess governance have been critiqued for their limitations as they largely depend on how and what you propose to measure ( Chhotray and Stoker 2009 ). Though such assessments provide valuable insights, the approach is somehow limited as it often fails to be explicit about the measurement ( Chhotray and Stoker 2009 ). This highlights that it is more important to identify what governance arrangements are considered appropriate for a particular context (prescriptive measures) than to judge the governance of a particular system (diagnostic measures) ( Chhotray and Stoker 2009 ).

The findings of this review could help to inform discussions among policy makers in countries considering governance as a mechanism to support health systems strengthening. Findings will help decision makers form a view on what governance is, and which principles are important in their context. Policy implementers at a more local level may choose and adapt one of the available frameworks or tools to assess governance and/or identify gaps in governance arrangements.

Conclusions

A variety of frameworks to assess health systems governance exist, but there are not many examples of their application in the literature. There is a need to validate and apply the existing frameworks and share lessons learnt regarding which frameworks work well in which settings to inform how existing frameworks can be adapted. A comprehensive assessment of governance could enable policy makers to prioritize solutions for problems identified as well as replicate and scale-up examples of good practice. Governance is not an ‘apolitical’ process, and there are no absolute principles that define governance; it is a diffuse concept that cuts across disciplines, and borrows from a range of social science theories. However, whether it is applied to health systems or political science, governance is concerned with how different actors in a given system or organization function and operate and the reasons for this. In the context of health systems governance, we believe a multidisciplinary approach to assessment is necessary.

This research was funded through the DFID/UKAid, Making it Happen programme.

Conflict of interest statement . None declared.

Supplementary Data

Supplementary data are available at HEAPOL online.

Supplementary Material

• Abimbola S, Olanipekun T, Igbokwe U. et al. 2015a. How decentralisation influences the retention of primary health care workers in rural Nigeria . Global Health Action 8 : 26616. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Abimbola S, Ukwaja KN, Onyedum CC. et al. 2015b. Transaction costs of access to health care: Implications of the care-seeking pathways of tuberculosis patients for health system governance in Nigeria . Global Public Health 10 : 1060–77. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Abimbola S, Negin J, Jan S, Martiniuk A.. 2014. Towards people-centred health systems: a multi-level framework for analysing primary health care governance in low-and middle-income countries . Health Policy and Planning 29 : ii29–39. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Abimbola S, Olanipekun T, Schaaf M, Negin J, Jan S, Martiniuk AL. 2016. Where there is no policy: governing the posting and transfer of primary health care workers in Nigeria . International Journal of Health Planning and Management . doi:10.1002/hpm.2356. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Avelino G, Barberia LG, Biderman C, Barberia LG.. 2013. Governance in managing public health resources in Brazilian municipalities . Health Policy and Planning 29 : 694–702. [ PubMed ] [ Google Scholar ]
• Baez-Camargo C. 2011. Accountability for Better Healthcare Provision: A Framework and Guidelines to Define Understand and Assess Accountability in Health Systems . Basel: Basel Institute of Governance. [ Google Scholar ]
• Baez-Camargo C, Jacobs E.. 2013. Social Accountability and Its Conceptual Challenges: An Analytical Framework . Basel: Basel Institute of Governance. [ Google Scholar ]
• Baez-Camargo C, Kamujuni P.. 2011. Final Report on Short Term Consulting Services Commissioned to the Basel Institute on Governance, Basel, Switzerland by the Swedish Embassy, Kampala, Uganda. Contract Number CO1533 Basel, Switzerland: Basel Institute on Governance.
• Baez-Camargo C, Jacobs E.. 2011. A Framework to Assess Governance of Health Systems in Low Income Countries . Basel, Switzerland: Basel Institute on Governance. [ Google Scholar ]
• Balabanova D, Mills A, Conteh L. et al. 2013. Good Health at Low Cost 25 years on: lessons for the future of health systems strengthening . Lancet 381 : 2118–33. [ PubMed ] [ Google Scholar ]
• Barnett-Page E, Thomas J.. 2009. Methods for the synthesis of qualitative research: a critical review . BMC Medical Research Methodology 9 : 59.. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Berlan D, Shiffman J.. 2012. Holding health providers in developing countries accountable to consumers: a synthesis of relevant scholarship . Health Policy and Planning 27 : 271–80. [ PubMed ] [ Google Scholar ]
• Brinkerhoff DW. 2004. Accountability and health systems: toward conceptual clarity and policy relevance . Health Policy and Planning 19 : 371–9. [ PubMed ] [ Google Scholar ]
• Brinkerhoff DW, Bossert TJ.. 2013. Health governance: principal-agent linkages and health system strengthening . Health Policy and Planning 29 : 685–93. [ PubMed ] [ Google Scholar ]
• Brinkerhoff DW, Bossert TJ.. 2008. Governance: Concepts, Experience and Program Options . Bethesda, MD: Health Systems 20/20 Project. [ Google Scholar ]
• Brinkerhoff DW, Goldsmith AA.. 2004. Good governance, clientelism, and patrimonialism: new perspectives on old problems . International Public Management Journal 7 : 163. [ Google Scholar ]
• Chhotray V, Stoker G.. 2009. Governance Theory and Practice: A Cross-Disciplinary Approach . London: Palgrave Macmillan UK. [ Google Scholar ]
• Cleary SM, Molyneux S, Gilson L.. 2013. Resources, attitudes and culture: an understanding of the factors that influence the functioning of accountability mechanisms in primary health care settings . BMC Health Services Research 13 : 320.. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Crombie IK. 1996. The Pocket Guide to Critical Appraisal . London: BMJ Publishing Group. [ Google Scholar ]
• Dixit A. 2009. Governance institutions and economic activity . American Economic Review . 99 : 5–24. [ Google Scholar ]
• Duran A, Saltman R.. 2015. Area of health policy and governance In: Kuhlmann E, Blank RH, Bourgeault IL, Wendt C (eds). The Palgrave International Handbook of Healthcare Policy and Governance . London: Palgrave Macmillan UK, 443–526. [ Google Scholar ]
• European Commission. 2008. Analysing and Addressing Governance in Sector Operations . Brussels: European Commission. [ Google Scholar ]
• Graham J, Amos B, Plumptre T. 2003. Principles for good governance in the 21st century: Policy Brief Number 15-August 2003 . Ottawa: The Institute on Governance. Available from: http://unpan1.un.org/intradoc/groups/public/documents/UNPAN/UNPAN011842.pdf .
• Health Systems 20/20. 2012. The Health System Assessment Approach: A How-To Manual. Version 2.0 . Washington DC: USAID; http://www.healthsystemassessment.com/health-system-assessment-approach-a-how-to-manual/ [ Google Scholar ]
• Huss R, Green A, Sudarshan H. et al. 2011. Good governance and corruption in the health sector: lessons from the Karnataka experience . Health Policy and Planning 26 : 471–84. [ PubMed ] [ Google Scholar ]
• Islam M. 2007. Health Systems Assessment Approach: A How-to Manual . Washington DC: United States Agency for International Development. [ Google Scholar ]
• Kaufmann D, Kraay A. 2007. Governance Indicators: Where Are We, Where Should We Be Going? The World Bank Research Observer . 23 :1–30.
• Kaufmann D, Kraay A, Zoido-Lobatón P.. 1999. Governance Matters . Washington DC: World Bank, Development Research Group, Macroeconomics and Growth, and World Bank Institute, Governance, Regulation, and Finance. [ Google Scholar ]
• Kirigia JM, Kirigia DG.. 2011. The essence of governance in health development . International Archives of Medicine 4 : 1–13. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kohler-Koch B, Rittberger B.. 2006. Review Article: The ‘Governance Turn’in EU Studies . JCMS: Journal of Common Market Studies 44 : 27–49. [ Google Scholar ]
• Lowenson R. 2008. Neglected Health Systems Research: Governance and Accountability . Geneva: Alliance for Health Policy and Systems Research and the World Health Organization; http://www.who.int/alliance-hpsr/AllianceHPSR_ResearchIssue_Governance.pdf [ Google Scholar ]
• Lucas P, Roberts H, Law C, Arai L, Baird J.. 2007. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews . BMC Medical Research Methodology 7 : 1–7. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McGuire M. 2010. Chapter 28: Network Management In: Bevir M. (ed). The SAGE Handbook of Governance . London: SAGE Publications, pp. 436–53. [ Google Scholar ]
• Mikkelsen-Lopez I, Wyss K, de Savigny D.. 2011. An approach to addressing governance from a health system framework perspective . BMC International Health and Human Rights 11 : 1–11. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Mutale W, Mwanamwenge MT, Balabanova D, Spicer N, Ayles H.. 2013. Measuring governance at health facility level: developing and validation of simple governance tool in Zambia . BMC International Health and Human Rights 13 : 9.. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• North DC. 1990. Institutions, Institutional Change and Economic Performance . Cambridge: Cambridge University Press. [ Google Scholar ]
• Olivier de Sardan JP. 2009. Researching the practical norms of real governance in Africa . London: Africa Power and Politics Programme. Retrieved from: http://www.institutions-africa.org/filestream/20090109-discussion-paper-5-researching-the-practical-norms-of-real-governance-in-africa-jean-pierre-olivier-de-sardan-jan-2009 .
• Ostrom E. 1990. Governing the Commons: The Evolution of Institutions for Collective Action . Cambridge: Cambridge University Press. [ Google Scholar ]
• Pierre J, Peters BG.. 2005. Governing Complex Societies: Trajectories and Scenarios . London: Palgrave Macmillan UK. [ Google Scholar ]
• Ramesh M, Wu X, He AJ.. 2013. Health governance and healthcare reforms in China . Health Policy and Planning 29 : 663–72. [ PubMed ] [ Google Scholar ]
• Schneider V. 2004. State theory, governance and the logic of regulation and administrative control In: Warntjen A, Wonka A. (eds). Governance in Europe: The Role of Interest Groups . pp. 25–41. https://www.utwente.nl/bms/pa/staff/warntjen/warntjen_wonka_governance_in_e.pdf [ Google Scholar ]
• Shukla M, Johnson Lassner K.. 2012. Governing for Health in Low- and Middle-Income Countries: Perspectives from the Field . Arlington, VA: Management Sciences for Health. [ Google Scholar ]
• Siddiqi S, Masud TI, Nishtar S. et al. 2009. Framework for assessing governance of the health system in developing countries: Gateway to good governance . Health Policy 90 : 13–25. [ PubMed ] [ Google Scholar ]
• Smith PC, Anell A, Busse R. et al. 2012. Leadership and governance in seven developed health systems . Health Policy 106 : 37–49. [ PubMed ] [ Google Scholar ]
• Stoker G. 2006. Why Politics Matters: Making Democracy Work . London: Palgrave Macmillan. [ Google Scholar ]
• Stoker G. 1998. Governance as theory: five propositions . International Social Science Journal 50 : 17–28. [ Google Scholar ]
• Vian T. 2008. Review of corruption in the health sector: theory, methods and interventions . Health Policy and Planning 23 : 83–94. [ PubMed ] [ Google Scholar ]
• Vian T, Bicknell WJ.. 2013. Good governance and budget reform in Lesotho Public Hospitals: performance, root causes and reality . Health Policy and Planning 29 : 673–84. [ PubMed ] [ Google Scholar ]
• Vian T, Brinkerhoff DW, Feeley FG, Salomon M, Vien NTK.. 2012. Confronting corruption in the health sector in Vietnam: patterns and prospects . Public Administration and Development 32 : 49–63. [ Google Scholar ]
• Williamson OE. 1999. The Mechanisms of Governance . Oxford: Oxford University Press. [ Google Scholar ]
• Williamson OE. 1979. Transaction-cost economics: the governance of contractual relations . The Journal of Law and Economics 22 : 233–61. [ Google Scholar ]
• Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R.. 2013.   RAMESES publication standards: realist syntheses . BMC Medicine 11 : 1.. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• World Bank. 2004. World Development Report 2004: Making Services Work for Poor People . Washington, DC: World Bank; https://openknowledge.worldbank.org/handle/10986/5986 [ Google Scholar ]
• World Bank. 1989. From Crisis to Sustainable Growth - Sub Saharan Africa: A Long-term Perspective Study . Washington, DC: World Bank; http://documents.worldbank.org/curated/en/1989/11/439705/crisis-sustainable-growth-sub-saharan-africa-long-term-perspective-study [ Google Scholar ]
• World Health Organization. 2007. Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes . Geneva: World Health Organization; http://www.who.int/healthsystems/strategy/everybodys_business.pdf [ Google Scholar ]

Grab your spot at the free arXiv Accessibility Forum

Help | Advanced Search

Computer Science > Computers and Society

Title: responsible ai governance: a systematic literature review.

Abstract: As artificial intelligence transforms a wide range of sectors and drives innovation, it also introduces complex challenges concerning ethics, transparency, bias, and fairness. The imperative for integrating Responsible AI (RAI) principles within governance frameworks is paramount to mitigate these emerging risks. While there are many solutions for AI governance, significant questions remain about their effectiveness in practice. Addressing this knowledge gap, this paper aims to examine the existing literature on AI Governance. The focus of this study is to analyse the literature to answer key questions: WHO is accountable for AI systems' governance, WHAT elements are being governed, WHEN governance occurs within the AI development life cycle, and HOW it is executed through various mechanisms like frameworks, tools, standards, policies, or models. Employing a systematic literature review methodology, a rigorous search and selection process has been employed. This effort resulted in the identification of 61 relevant articles on the subject of AI Governance. Out of the 61 studies analysed, only 5 provided complete responses to all questions. The findings from this review aid research in formulating more holistic and comprehensive Responsible AI (RAI) governance frameworks. This study highlights important role of AI governance on various levels specially organisational in establishing effective and responsible AI practices. The findings of this study provides a foundational basis for future research and development of comprehensive governance models that align with RAI principles.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Smart City Governance in Developing Countries: A Systematic Literature Review

Smart cities that make broad use of digital technologies have been touted as possible solutions for the population pressures faced by many cities in developing countries and may help meet the rising demand for services and infrastructure. Nevertheless, the high financial cost involved in infrastructure maintenance, the substantial size of the informal economies, and various governance challenges are curtailing government idealism regarding smart cities. This review examines the state of smart city development in developing countries, which includes understanding the conceptualisations, motivations, and unique drivers behind (and barriers to) smarty city development. A total of 56 studies were identified from a systematic literature review from an initial pool of 3928 social sciences literature identified from two academic databases. Data were analysed using thematic synthesis and thematic analysis. The review found that technology-enabled smart cities in developing countries can only be realised when concurrent socioeconomic, human, legal, and regulatory reforms are instituted. Governments need to step up their efforts to fulfil the basic infrastructure needs of citizens, raise more revenue, construct clear regulatory frameworks to mitigate the technological risks involved, develop human capital, ensure digital inclusivity, and promote environmental sustainability. A supportive ecosystem that encourages citizen participation, nurtures start-ups, and promotes public-private partnerships needs to be created to realise their smart city vision.

Related Papers

Smart Cities Transforming Developing Nations for a Sustainable Future

Bassem Fahmy

The Developing Countries Smart Cities Framework manifesto underscores the significance of integrating various components to create a unified urban environment. Smart utilities and infrastructure leverage innovative technologies for optimised service delivery, while smart institutions and governance mechanisms ensure transparent and efficient urban planning. Economic development thrives through smart enterprise initiatives and public-private partnerships, fostering growth and innovation. Integrating top-down and bottom-up strategies, a people-centric approach prioritises residents' needs and aspirations and promotes social cohesion and community engagement. This inclusive model cultivates a vibrant and resilient urban society.

Telecommunications Policy

Kris Hartley

Smart city initiatives have been researched primarily in the developed country context. In developing countries, however, emerging technologies are enabling progress on urban functionality, productivity, and livability. A deeper understanding of facilitative policy conditions unique to developing countries would be useful to both theory and practice. This study presents empirically grounded insights about the policy implications of smart city development in developing countries, based on surveys of experts from the public and private sectors in 10 Vietnam cities. The study makes three contributions. First, it provides new evidence that pursuing smart city development (SCD) is not a mere alternative but a crucial strategic imperative. While facing persistent problems, Vietnam's cities exhibit significant and rapidly improving readiness for SCD. Second, the study provides new insights into related policy issues and challenges, including the positive link between e-government development and control of corruption, the risk of bias toward operational management over institutional reform, and the lack of a clear development strategy. Finally, the study proposes a model for guiding smart city initiatives in developing countries.

Dr Subashini Suresh

Contemporary Social Science

Mohammed Agbali

The Smart City (SC) concept has emerged as an innovative response to the challenges and opportunities created by rapid urbanisation around the world. Consequently, city governments around the globe are embracing ‘Smart’ strategies as part of major steps towards making their cities more livable and sustainable. Despite global attention on the SC concept and its recognised potential to improve the circumstances of cities in any region, there is paucity of research on appropriate framework models to assess the impacts of smartness on cities in developing countries. Based on a systematic literature review and a pilot study with key SC stakeholders, the study initially built a conceptual framework for a multi-dimensional understanding of the critical success factors (CSFs) and indicators of Smart Cities (SCs) that aligned with stakeholders’ perceptions and experiences of SCs. Then, the study examined Smart Innovations in Boston, FCT-Abuja, and Manchester as case studies of SC development...

Zaigham Mahmood

Cities have been identified as carriers of development activities worldwide. Unfortunately, they are faced with a series of constraints. One among them is high population density in cities. In the next several years, 58.2 percent of the world and 75.8 percent of the European population will live in urban areas. According to current projections, by 2050 two-thirds of the world population will live in cities. Cities are confronted with constant need to increase energy consumption, transportation, buildings, public spaces, etc. All of this points to the need for “smart” solutions that ensure economic prosperity and social wellbeing for cities’ citizens. The most efficient way to achieve this is by mobilizing cities’ resources and stakeholders using new technologies and policies. In the literature this concept is known as the Smart City concept. It contains several smart dimensions that relate to smart economy, smart people, smart governance, smart mobility, smart environment, and smart...

Jamile Sabatini-Marques , Eduardo Moreira da Costa

The convergence of technology and the city is commonly referred to as the 'smart city'. It is seen as a possible remedy for the challenges that urbanisation creates in the age of global climate change, and as an enabler of a sustainable and liveable urban future. A review of the abundant but fragmented literature on smart city theories and practices, nevertheless, reveals that there is a limited effort to capture a comprehensive understanding on how the complex and multidimensional nature of the drivers of smart cities are linked to desired outcomes. The paper aims to develop a clearer understanding on this new city model by identifying and linking the key drivers to desired outcomes, and then intertwining them in a multidimensional framework. The methodological approach of this research includes a systematic review of the literature on smart cities, focusing on those aimed at conceptual development and provide empirical evidence base. The review identifies that the literature reveals three types of drivers of smart cities—community, technology, policy—which are linked to five desired out-comes—productivity, sustainability, accessibility, wellbeing, liveability, governance. These drivers and outcomes altogether assemble a smart city framework, where each of them represents a distinctive dimension of the smart cities notion. This paper helps in expanding our understanding beyond a monocentric technology focus of the current common smart city practice.

Ayyoob Sharifi

Despite the wealth of research on smart cities, there is a lack of studies examining interlinkages between smart cities and Sustainable Development Goals (SDGs). In other words, there is limited research on how implementing smart city solutions can lead to co-benefits and/or trade-offs for achieving SDGs. This systematic literature review was conducted to fill this gap. Results show that responsible development/implementation of smart city solutions and technologies could contribute to the progress toward SDGs. The literature is mainly focused on SDG 11 (Sustainable Cities and Communities), SDG 12 (Responsible Consumption and Production), SDG 7 (Affordable and Clean Energy), and SDG 6 (Clean Water and Sanitation). More work on other SDGs is needed. There is a bias toward reporting the benefits of smart cities. These include accelerating economic growth, improving efficiency, strengthening innovation, and raising citizen awareness. These benefits indicate that smart cities could catalyze the transition to sustainable development and address climate change challenges. However, this requires addressing trade-offs related to issues such as privacy and cyber security, costs of infrastructure upgrading, rebound effects associated with efficiency improvements, biased decision-making, reproduction of social biases, digital divide and lack of skills, misuse of AI, and limited legal setup. This review elaborates on these trade-offs and offers solutions to minimize them. Results show that the COVID-19 pandemic has increased attention to the interactions between smart cities and the SDGs, particularly those related to health and climate action. However, it has also cemented many new unethical practices. This review highlights governance/policy challenges that should be addressed to ensure smart cities can better contribute to the SDGs. It concludes that multi-scale and transparent governance mechanisms and regulatory frameworks are crucial for ensuring that smart city solutions support the transition toward sustainable and resilient cities.

International Review of Administrative Sciences

Manuel Pedro Rodríguez Bolívar

Academic attention to smart cities and their governance is growing rapidly, but the fragmentation in approaches makes for a confusing debate. This article brings some structure to the debate by analyzing a corpus of 51 publications and mapping their variation. The analysis shows that publications differ in their emphasis on (1) smart technology, smart people or smart collaboration as the defining features of smart cities, (2) a transformative or incremental perspective on changes in urban governance, (3) better outcomes or a more open process as the legitimacy claim for smart city governance. We argue for a comprehensive perspective: smart city governance is about crafting new forms of human collaboration through the use of ICTs to obtain better outcomes and more open governance processes. Research into smart city governance could benefit from previous studies into success and failure factors for e-government and build upon sophisticated theories of socio-technical change. This arti...

Corporate and business strategy review

PAPON MOOLNGEARN

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

International Journal of Sustainable Development and Planning

Sarbeswar Praharaj

International Journal of Public Sector Management

Francesca Culasso

Meredian Alam

Administratio Publica

Gerrit Van der Waldt , Mukundi Maphangwa

Lecture Notes in Computer Science, vol 13117

Nkhangweni Mashau

Vasiliki Geropanta

Research in Globalization

Runlin Yang

Construction economics and building

Ayodeji E Oke

Journal of Urban Technology

stan geertman

Technology Analysis & Strategic Management

Gilmar Masiero

Proceedings of the 24th Conference on Computer Aided Architectural Design Research in Asia (CAADRIA) [Volume 2]

Victor Calixto

Malaysian Townplan Journal

Seng Boon Lim , Jalaluddin Abdul Malek

2019 3rd International Conference on Smart Grid and Smart Cities (ICSGSC)

Cecilia Tortajada

Smart city technology: a potential solution to Africa's growing population and rapid urbanization

Adaku Jane Echendu

Si Ying Tan

bunga hidayati

Annals of Spiru Haret University. Economic Series

Abolaji Atobatele

E3S Web of Conferences

Dyah Mutiarin

Emerald Publishing Limited

Agboola Oluwagbemiga (Ph.D.)

JPPUMA Jurnal Ilmu Pemerintahan dan Sosial Politik Universitas Medan Area

Achmad Nurmandi

Caroline E COVELL

How the smart governance model shapes cities? Cases from Europe

Demokaan DEMİREL , Maksud Emre Mülazımoğlu

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• DOI: 10.1080/14719037.2019.1648697
• Corpus ID: 202327174

From government to governance…to meta-governance: a systematic literature review

• J. Gjaltema , R. Biesbroek , K. Termeer
• Published in Public Management Review 1 December 2020
• Political Science

86 Citations

Meta-governance as partial organization.

• Highly Influenced

The many shapes and sizes of meta-governance. An empirical study of strategies applied by a well-advanced meta-governor: the case of Dutch central government in education

The existential stakes of platform governance: a critical literature review, a (meta)governance framework for multi-level governance of inter-organizational project networks, state-centred-collaborative-governance: a “new” governance model for ict success, the networked bureaucracy: reinventing formalization in the context of collaborative governance, from local government to local governance: a systematic literature review and future research agenda, new paths for public governance: literature review, content analysis, and conceptual framework proposal in an integrative view, the existential stakes of platform governance and online content regulation: a critical conceptual model, framing metagovernance in the context of developing democracies: an institutionalist viewpoint, 70 references, mechanisms of private meta-governance: an analysis of global private governance for sustainable development, the politics of meta-governance in transnational private sustainability governance, experiencing multi-level meta-governance, governing collaboration: practices of meta-governance in finnish and norwegian local governments, meta-governance: values, norms and principles, and the making of hard choices, making governance networks effective and democratic through metagovernance, more micro than meta competing concepts of metagovernance in the european union, meta-governance of regional governance networks in nordic countries, understanding governance: ten years on, the cultural dimension of metagovernance: why governance doctrines may fail, related papers.

Showing 1 through 3 of 0 Related Papers

• Systematic Review
• Open access
• Published: 17 August 2024

Direct and indirect effects of economic sanctions on health: a systematic narrative literature review

• Vahid Yazdi-Feyzabadi 1 ,
• Atefeh Zolfagharnasab 2 ,
• Soheila Naghavi 3 ,
• Anahita Behzadi 1 ,
• Maysam Yousefi 4 &
• Mohammad Bazyar   ORCID: orcid.org/0000-0003-2543-1862 5  

BMC Public Health volume  24 , Article number:  2242 ( 2024 ) Cite this article

43 Accesses

Metrics details

Economic sanctions are defined as restrictions imposed by other countries against individuals, groups, or governments of other countries. These sanctions have a detrimental impact on the economies of countries and can also limit access to healthcare services for people as a secondary consequence. This study aims to systematically review the literature to examine the direct and indirect effects of economic sanctions on health through a narrative synthesis.

This systematic literature review was limited to papers published between January 1990 and July 2023. Relevant documents published in English and Persian were searched for in databases including Cochrane Library, PubMed, Embase, Scopus, Web of Science, SID, Magiran, and Irandoc. The direct and indirect effects of sanctions on health were classified using two frameworks proposed by the World Health Organization (WHO): the Health System Building Blocks and “Social Determinants of Health”.

Out of a total of 18,219 articles, 59 were selected based on inclusion criteria. The effects of sanctions were divided into direct and indirect groups. Direct effects encompassed seven main themes: access to essential medicine, medical products, vaccines and technologies; financing; health workforce; service delivery; research and health information systems; health outcomes; and financial risk protection. Indirect effects also were classified into six main themes: socioeconomic status; food and agricultural products; stress; early life conditions; high-risk behaviors and addiction; and transport. Most studies focused on the access to medicines, food, economic and social status.

Conclusions

Economic sanctions have had profoundly negative impacts on all aspects of the healthcare system. The international community must address these effects on health and take necessary measures to prevent or mitigate them, particularly in ensuring the provision of basic and essential healthcare needs for individuals and communities.

Peer Review reports

Sanctions are purposeful and determined restrictions imposed by one or more countries against another individual, group or countries’ government. Sanctions are usually imposed by international organizations as a pressure tool for responding to the course of actions of any country that opposes them [ 1 ].

Economic sanctions are the most common type of these restrictions. The two main types of these sanctions are trade and financial restrictions. Trade sanctions restrict imports to and exports from the countries under sanctions while financial sanctions are closely related to economic ones, but their focus is on banning the money flows and financial resources into or out of the country. These sanctions can include blocking government assets, restricting access to financial markets, loans and credits limitations, restricting international financial exchange, and also sales and trade abroad [ 2 ].

Economic sanctions reduce people’s access to basic necessities of life by debilitating the economic situation, decreasing welfare and weakening the functions of the target country’s social systems. One of the most important areas affected through these boycotts is health. Due to the expansion of health scope, these limitations affect different parts of health system itself and as a result endanger people’s life [ 3 , 4 ].

Studies in various countries, including Iran, Iraq, Cuba, Yugoslavia and Haiti, discussed the effects of sanctions on health. In Haiti, economic sanctions have reduced incomes, increased unemployment and poverty along with mortality by 1 to 4 years, and destroyed families [ 5 ]. In Iran, especially in healthcare area, sanctions have resulted in increasing the cost of essential procedures and drugs such as diagnostic procedures for cancers and chemotherapy drugs. The difficulties in getting required licenses for financial transactions and transportation insurance due to sanctions has left the country with a shortage of drugs and health equipment [ 6 ].

Sanctions have devastating effects on the health of vulnerable patients or health systems customers too. Patients who are suffering from diseases such as asthma, thalassemia, hemophilia, chronic diseases, blood disorders, multiple sclerosis and HIV/AIDS have limited access to drugs [ 7 ]. While comparing, in developed countries mortality rates decreased using appropriate drugs [ 8 , 9 ].

Different countries may use broad policies to prevent or adjust the negative effects of economic problems on health systems, although these policies may not be successful in ensuring continued access to health services [ 10 , 11 ].

Although sanctions may be designed for excluding medical products from the list, they can still have an inevitable impact on access to health services. Thus, the imposition of economic sanctions can threaten public health directly [ 12 ].

Furthermore, economic sanctions suppress the health indirectly by adversely impacting on other related parts known as social determinants of health (SDH) and Sustainable Development Goals (SDGs). Economic sanctions impact all aspects of the social determinants of health (SDH) framework, leading to negative effects on health equity and well-being. Sanctions can alter social and political systems, such as governance, labor markets, education, trade, housing, and redistributive policies, influencing people’s health. Structural determinants like income, education, and occupation are affected by sanctions, changing health opportunities and status, especially for the economically disadvantaged. Intermediary determinants, including material and psychosocial circumstances, are also influenced negatively by sanctions. For instance, housing quality declines post-sanctions due to increased costs of land and materials, while food consumption patterns shift towards cheaper, less nutritious options. Sanctions create psychosocial stressors like job insecurity and uncertainty, leading to frustration and stress [ 13 , 14 , 15 , 16 ].

Continued sanctions may hinder countries’ progress towards achieving Sustainable Development Goals (SDGs), particularly SDG-3 for healthy lives and well-being. Economic stability is crucial for meeting health-related SDGs, and any failures in this regard would disproportionately impact citizens in targeted countries. In the context of developing countries, where progress toward SDGs is often hindered by limited resources and systemic disparities, the impact of economic sanctions on health systems and overall well-being is profound. SDGs, with their emphasis on health (Goal 3) and the overarching aim of leaving no one behind, seek to address disparities and ensure equitable access to healthcare services. Economic sanctions, however, disrupt this delicate balance, exacerbating existing inequalities and impeding the ability of nations to meet the health-related targets outlined in the SDGs [ 13 ].

Given that, the effects of sanctions depend on the situation of countries and vary from one to another, there is no complete evidence of a comprehensive impact of sanctions on different part of society’s system especially in health despite of its importance. To comprehensively address these issues, a rigorous examination of evidence through narrative systematic reviews becomes imperative. This study aims to provide a detailed narrative synthesis of the direct and indirect effects of economic sanctions on health system building blocks and public health focusing on social determinants of health, thereby contributing to a better understanding of the broader consequences of such measures.

The following steps were taken to review literature systematically [ 17 , 18 ].

Research question

The main question that we wanted to answer in this study was to investigate and categorize the effects that economic sanctions impose on health directly and indirectly.

Search strategy and identifying literature

This systematic review was carried out according to the latest version of PRISMA guidelines [ 19 , 20 ]. For the purposes of the study, following databases were searched by one of the authors experienced in systematic research: Cochrane Library, PubMed, Embase, Scopus, Web of Science, SID, Magiran, Irandoc. The search strategy (see Additional file 1 for an example) was first devised for use in PubMed and subsequently adapted for the other databases. The search was limited to papers published between January 1990 and July 2023 and to studies involving economic sanctions independently as a hard power exercise. Other hard power exercises to achieve foreign policy goals such as war and conflicts were excluded. We selected the appropriate keywords from studying similar studies, discussion among research team and intended frameworks for extracting the data. The search term “sanctions” and “public health” were used for PubMed; terms associated with “economic sanctions” and “public health” were used for the title or abstract in the other databases if required (MeSH term; major focus and/or exploded depending on the database). In brief the following terms were searched using Boolean operators: sanction, embargo, health, human resource, medical instrument, medicine, pharmaceutics, disease, mortality, medical equipment, medical devices, drug, health care, Taskforce, health personnel, health workers, morbidity, illness, and food.

Screening and article selection criteria

Duplicate results were removed after searching the databases using Endnote software version X8. After removing the duplications, a screening of publications, based on titles and abstracts was performed by two researchers independently. In second screening, then, the suspected documents were re-examined by a third person from the research team to decide whether to enter or not.

As the final step of screening, the full texts of the remaining publications were independently assessed for inclusion by pairs of reviewers once more and any potential disagreements were resolved through consensus and if necessary by the third opinion from the research team.

The articles not meeting the below criteria were excluded:

Articles published in languages other than English and Persian.

Articles available in preprint servers.

Articles did not match the question and objectives of the research like those related to the effects of wars and conflict on health.

Conference abstracts, books, reports and dissertations.

Records not in line with the quantitative, qualitative and mixed-method original articles including letter to Editor, commentary, opinion/viewpoint/perspective.

Articles published before 1990.

After reaching the final list of studies to be reviewed thoroughly, we supplemented our database search by screening bibliographic of chosen articles to identify any additional relevant publications. The bibliographic of other relevant systematic articles were also searched actively for retrieving other missing articles.

Data extraction

After finalizing the final list of articles, the full text of the selected articles were studied precisely and required information was extracted. In order to capture the maximum available evidence regarding the effects of economic sanctions, no quality assessment was employed in our systematic literature review. This approach allowed us to include a wide range of studies, regardless of their methodological quality, thus providing a comprehensive overview of the existing literature. This method is consistent with approaches used in narrative synthesis where the primary aim is to summarize broad evidence on a topic rather than critically appraise each study’s quality. The extracted information was divided into two sections. The first one, consisting the bibliographic information included the title of article, the year of publication, the first author, and the title of the journal and the second section reports the frequency of articles according to the main topics addressed in their results.

Data analysis and presenting results

For identifying key concepts and main themes, each of selected articles studied carefully. After completing the data extraction table, the researchers shared the concepts with other members of the research team, and agreement was reached. As many other factors outside the borders of health system affects the health, generally known as social determinants of health (SDH), we applied two common popular frameworks to categorize the direct and indirect effects impacts of sanctions on health system and public health. To address the direct impact of sanctions on health, Health System Building Blocks framework proposed by World Health Organization (WHO) was proposed which consists of six key components including “service delivery”, “health workforce”, “health information systems”, “access to essential medicines”, “financing” and “government/ leadership”. This framework also covers intermediate (e.g. access, coverage, quality and safety) and four final goals including Improved health (level and equity), Responsiveness, Social and financial risk protection, and Improved efficiency [ 21 , 22 ].

To cover other effects of sanctions occurring in other sections beyond the health system but affecting health indirectly, the approach of “Social Determinants of Health” was applied which comprises of the following 10 elements, “The social gradient”, “Stress”, “Early life conditions”, “Social exclusion”, “Work”, “Unemployment”, “Social support”, “Addiction”, “Food”, and “Transport“ [ 23 ].

Search process

A total number of 18,219 articles were identified, which after removing the overlaps, 12,838 articles remained. Following the initial review of the title and abstract of all retrieved articles, a further 12,439 articles were excluded. Out of 399 records, the full text of 390 articles were retrieved and evaluated for eligibility.

After a final review, 331 articles were excluded due to not intended study design or not addressing the question and aims of the current research. Finally 59 research articles were included in the study (Fig.  1 ). A summary of included studies’ features is reported in Table  1 .

The PRISMA algorithm of study selection process

Study features

The study information collected from 11 countries which included Iran, Iraq, Cuba, Syria, Haiti, Yugoslavia, Lebanon, Serbia, Nicaragua, Sri Lanka, Russia and South Africa. Iraq [ 4 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 ] and Iran [ 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ] had the maximum number of studies. Most of the studies were descriptive or analytical. Four were qualitative and also three studies were designed as a mix-method. The share of different regions from economic sanctions studies is shown in Fig.  2 .

The different regions share of economic sanctions studies

Study areas

Included studies have examined the impact of sanctions in various areas. The amount of available and accessible information about the effect of sanctions varied from one area to another. Most studies addressed the impact of economic sanctions on access to medicine or food and also socioeconomic status.

Almost half of studies mentioned the effect of sanctions on access to drugs, these studies covered approximately all countries targeted by sanctions. More than a quarter, discussed the socioeconomic situation. Food access and malnutrition were also explained in about another quarter of the articles. According to studies, the vulnerable groups which affected most by sanctions were the poor, patients, women and children. The proportion of different parts of health system and different social health determinants affected by economic sanctions is reported in Table  2 ; Fig.  3 .

The proportion of different parts of health system and social determinants of health mentioned in retrieved studies which are affected by economic sanctions

The Table  2 provides an overview of the frequency of direct and indirect effects of sanctions on health, based on findings from 59 selected articles. The data is categorized into direct effects on the health system and indirect effects on population/public health, highlighting both the immediate and broader consequences of sanctions. The most frequently mentioned direct effect was the impact on access to essential medicine, medical products, vaccines, and technologies, with 22 documents, accounting for 37.3% of the papers. On the other hand, the most frequently cited indirect effect was on the socioeconomic situation, mentioned in 9 documents (15.2%).

Themes and sub-themes

The effects of sanctions on health were categorized into two broad direct and indirect groups. Following the WHO’ Health System Building Blocks, direct effects include 7 main themes as followed: access to essential medicine, medical products, vaccines and technologies; financing; health workforce; service delivery; research and health information systems; health outcomes; and financial risk protection (Table  3 ). Indirect effects also were summarized in 6 main themes consisting: socioeconomic status; food and agricultural products; stress; early life conditions; high-risk behaviors and addiction; and transportation (Table  4 ).

Direct effects

Access to medicines, medical products, vaccines, and technologies.

Access to medicine is one of the main goals of health systems. Numerous studies have been reported on drug shortages and public concerns about patients’ difficulties for getting their essential’s [ 46 ]. The findings related to access to medicine were divided further into three sub-themes: reduced access to imported raw materials, decreased access to imported or foreign drugs, and increased drug prices.

There are some findings indicated that sanctions prevent the import of essential medical supplies [ 29 , 36 , 44 , 47 , 48 ]. Therefore, the manufacture of local drugs is affected and access to them is reduced.

For example, Iran experienced a significant decrease in access to asthma drugs which produced locally in Iran, because of local producers relied on imported raw materials [ 36 ]. In Yugoslavia, as a result of imposing restrictions on pharmaceutical industry, the available essential drugs decreased by more than 50% [ 47 ]. Syria also faced a shortage of raw materials for producing domestic drugs for heart disease, cancer and diabetes [ 48 ]. In a similar way, in Iraq, the provision of laboratory services reduced because of raw chemicals shortage [ 29 ].

Limited access to imported drugs was another direct effect [ 49 ]. The shortage of essential medicines in countries suffering from sanctions was a main concern and access to such medicines including chemotherapy, chronically illness treatments, psychiatric services, MS and antiepileptic drugs was limited considerably [ 29 , 31 , 36 , 37 , 40 , 42 , 44 , 47 , 50 ]. Access to hemophilia and thalassemia drugs was severely affected too [ 39 ]. Problems caused by economic sanctions also affected the pharmaceutical market which as a result, lead to an sharp increase in the prices [ 5 , 42 , 51 ].

Studies showed that economic sanctions reduced the import of and access to medical equipment to great extent [ 3 , 24 , 29 , 52 ]. In Cuba, the number of X-rays decreased by 75% [ 3 ]. Many American companies refused to sell drugs or medical equipment assigned for Nicaragua. Severe shortage of medical products in health system became apparent in 1985 and worsened in 1986 [ 24 ].

Also, studies revealed that economic sanctions have reduced access to vaccines and caused less immunization against infectious diseases [ 25 , 53 ].

In case of Cuba, the country’s ability to produce chlorine decreased and the number of populations with no access to safe drinking water increased, therefore population covered by chlorine water systems decreased from 98% in 1988 to 26% in 1994 [ 3 ].

Health financing

Health financing counts as essential ability of health systems to maintain and improve the community well-being. The economic crisis is affecting the financial capacity of health care system and has hampered the financial support for providing health services [ 52 , 54 ]. During the economic sanctions, budget constraints also prevented some health care programs from being fully implemented [ 31 ].

Health workforce

A study done in Iraq showed that economic sanctions resulted in widespread expulsions of health care professionals, while many of them were belong to foreign nationals. Also, physicians had to do a lot of extra work, along with increasing pressures which caused them.

leave their jobs behind [ 29 , 31 ].

Health services delivery

The imposition of economic sanctions, resulted in labor shortages, limited access to medical equipment, affecting the process of providing health services and made it worse. This imposed much more pressure on the ability of health system as whole particularly during crises like the COVID-19 pandemic [ 55 ]. Meanwhile various studies showed a reduction in quantity and quality of provided services too [ 5 , 29 , 31 , 52 , 56 , 57 ]. A study in Iran showed that due to economic sanctions, from 18 brachytherapy centers in 2018, only two centers were usable since 2015 and also the gap between Iran’s available facilities for radiation therapy and international standards deepened [ 58 ]. According to the Program of Action for Cancer Therapy, sanctions have disrupted Iran’s National Cancer Control Program (NCCP) as they have influenced all phases of treatment from prevention, to diagnosis/treatment, palliative care, monitoring, and also technology and drug availability [ 59 ].

Research and health information systems

The effects of economic sanctions on research and health information systems were divided into three sub-themes: reduced access to scientific resources and virtual sites, disruption of international interactions and conferences, and restricted research activities. Sanctions also limited access to scientific magazines and books [ 29 , 52 ]. Specialists were unable to get visas and travel abroad to attend international conferences, which reduced scientific exchanges [ 52 ].

The severe financial pressure of sanctions intensified restrictions of scientific travels and communications with the outside world led to a lack of access to educational materials and global medical advances [ 31 ].

Economic sanctions also had negative effects on both research and science production activities, including smaller scientific communication, difficulties in research processes, and consequently, decline in the quality along with quantity of research and science development activities [ 14 , 45 , 60 ].

Health outcomes and financing risk protection

International evidence about the UN sanctions indicates that they reduces life expectancy by about 1.2–1.4 years on average. It was also shown that this reduction is much more severe in vulnerable groups of society like women. This lower life expectancy in the studied countries occurred to great extent due to higher child mortality and Cholera deaths and also spending less amount of public budget on health care [ 61 , 62 ]. Studies from Iran show that multiple sclerosis patients faced higher out-of-pocket payments, catastrophic health expenditures and the poverty index [ 63 ]. Similarly, studies from other countries show higher mortality rate from infectious diseases and more difficulties for optic and neuropathic patients [ 64 , 65 ]. Physical rehabilitation experts in Iran also concern about high price that people with physical problems have to pay for prostheses which in turn have negative consequences for practitioners themselves [ 66 ].

Indirect effects

The effects of economic sanctions are not targeted and they also influence sectors other than health which can affect general health indirectly. These effects can be categorized under a general concept as social determinants of health (SDH). The main SDHs extracted from the retrieved studies are as follows:

Economic and social status

The main target of economic sanctions is the economy and money flows of countries which had negative consequences for countries’ economy themselves and other related areas [ 67 ]. The indirect outcomes in the area of Economic and Social Status were categorized into 6 sub-themes including: rising unemployment, decreasing income, declining welfare, increasing poverty, trade barriers, rising prices and decreasing purchasing power.

Majority of studies in Iran, Iraq, Cuba, Haiti, Syria, South Africa addressed the effects of sanctions on the socioeconomic situation [ 4 , 5 , 29 , 44 , 47 , 48 , 52 , 68 ]. These sanctions banned and reduced the exports of products which in turn caused unemployment among those who relied on importing such products to make money. Unemployment rose sharply in Haiti with the cessation of mango exports, on which many poor people depended [ 47 ]. Also, in this country some of factories such as clothing, sports and assembly, reduced the number of workers, which was accompanied by rising the rate of job loss [ 5 ].

Continuing this situation, economic problems became more and more prevalent. In Haiti, many people lost their main source of income [ 5 ]. In Iraq, wage fell and there was hardly enough to buy the necessities of daily life [ 29 ]. A review of studies during this period revealed that the reason of increasing social problems and the disintegration of many family structures was the fall in incomes [ 5 , 44 ].

On the other hand, poverty increased as soon as economic problems intensified. Some of the middle classes’ families were forced to sell their houses and apartments [ 29 ]. Also, school enrollment declined due to the poverty [ 5 ].

Another important effect was trade barriers so that reduced the rate of investment and the number of foreign companies. The number of active American companies in South Africa fell from 267 in 1986 to 104 in 1991 [ 47 ]. Loss of markets, credits, and favorable trade conditions, devaluation of the national currency against the US dollar, the oil exports stoppage, alongside the reduction of basic goods imports were among the other effects [ 29 , 47 , 48 ]. On the other hand, prices increased while purchasing power decreased [ 4 , 5 , 48 , 52 ].

Food and agriculture

Numerous studies in Iran, Cuba, Iraq, and Haiti have shown that economic sanctions reduced food imports while increased their prices, and restricted proper diet 41) [ 3 , 5 , 24 , 29 , 47 ]. In Cuba, food imports decreased by almost 50% from 1989 to 1993 as a result of falling rate of imports while shifting to low-quality protein products which posed serious threats on population’s health. In Haiti, staple food prices increased fivefold from 1991 to 1993 [ 47 ]. Likewise, the prices of all food groups increased significantly in Iran in 2018 due to the limitations in international financial exchanges, right after the re-imposition of sanctions. The price increase was higher in vegetable, meat, and fruit groups which made it nearly impossible to follow a healthy diet [ 69 ].

Prices of basic commodities such as wheat, rice and sugar rose in Iraq, too [ 29 ]. On the other hand, the lack of foods containing B vitamins group in Cuba led to the epidemic of neuropathy [ 47 ]. Poor nutrition among pregnant women in Iraq increased anemia [ 24 ]. Meal and per capita protein intake decreased [ 3 , 5 ]. At the same time malnutrition also increased during restrictions [ 5 , 35 , 47 , 70 ] and furthermore caused reduction in crop production and agricultural support [ 5 , 71 ]. Other studies also revealed that availability and stability were the most affected dimensions of food security following imposing economic sanctions [ 72 ]. A study about the impact of the UN and US economic sanctions on the environment in Iran found that while these sanctions initially improved Iran’s environment in the short term, they had long-term damaging effects [ 73 ].

Economic sanctions exacerbate stressful conditions. According to studies, increased fear and uncertainty, and increased mental health problems are among the negative effects of sanctions in this category [ 5 , 31 , 48 , 53 , 74 ].

Early life conditions

A good start in life means supporting mothers and young children. A study found the exposure to adverse economic conditions in infancy and early childhood was effective in long-term negative health outcomes [ 75 ].

High-risk and addictive behaviors

People turning to high-risk behaviors and addiction along with their consumption patterns can be affected and intensified by economic and social conditions. According to the findings, the effects of economic sanctions in case of risky and addictive behavior were categorized into two sub-groups consist of increasing high-risk behaviors and addiction.

As evidences revealed, economic sanctions increased suicide and violence. Studies shown that the rate of deaths caused by violence and suicides have increased in Yugoslavia and Cuba during limitation periods [ 3 , 47 ]. In Haiti, charges against children, criminal conspiracy, robbery, and drug use were much more serious [ 5 ] and this happened along with another important result which was changing in drug use patterns and increased drug abuse problems.

The common use of syringes for drug injection has increased, posing a risk to abusers. Due to economic problems, people entered mass drug distribution networks and drug trafficking to make money; tried steal or other illegal ways to earn money for buying or supplying drugs. Rising drug prices have led to the neglect family economic basket and reduced attention to the factors such as education and health care, which have resulted in low quality of life for consumers and their families [ 41 ].

According to the studies, economic sanctions in different countries affected communities’ health in different ways. Economic sanctions are supposed to force a country’s government to reconsider its policies by putting and imposing economic pressure. Although they should not target humanitarian goods, studies in various countries have revealed their direct and indirect effects on community’s health and threats for people’s right to health. The most important effects were found in the access to medicine and change in socio-economic conditions, while ensuring access to medicines for people who needed them, is one of the most emphasized goals of health systems all around the world [ 76 ]. Clearly economic sanctions suppress economic growth of the targeted countries in different ways and the lower economic situation can influence all aspects of the whole community and people’ life including their health status directly and indirectly. The World Bank data confirm that sanctions reduced Iran’s economic growth by 38% within three years, as GDP per capita dropped from US$ 7833 in 2012 to US$ 4862 in 2015. Moreover, unemployment increased from 10.4% in 2013 to 13.1% in 2017, and the economic inequality in household expenditure, measured by Gini coefficient, increased from 37 to 41%, since 2012, due to economic sanction. Clearly this economic inequality can lead to health inequity in population [ 77 ]. When the economic situation worsens in general, the financial capacity of health system and also the financial power of people will be affected. Evidence from different studies proved that the general budget of health decreased and out-of-pocket payments increased especially for those patients who depend on foreign and imported drugs [ 54 , 63 ].

In the present study, surveys in different countries showed that economic sanctions through devaluing the national currency, affected access to health goods and services, including drugs and medical equipment. Countries depending on drug technology, requiring the imported raw materials, experienced a severe restriction for accessing to medicine and drugs. On the other hand, these limitations caused a sudden increase or inflation in the prices of medicine and equipment. This effect would be worse for people who suffer from chronic diseases and are unable to purchase or use health care services [ 16 ]. For example, sanctions in 2011 caused a 14 times increase in the price of formula in Iran for infants suffering from food allergies. Besides that, uncertainty about the availability of drugs following sanctions also changes the behavior of people as the stored formula for infants not needing a specific formula which was enough for 2 months was distributed only within 4 days in September, 2018 [ 78 ].

This is while the economic sanctions reducing the power of supporting health services by limiting budgets and funds. Health financing is essential for the ability of health systems to maintain and improve human health through keeping them capable to fund and provide health services. Without the necessary funding, no health workers will be hired, no medication will be available, and as the same way, no health promotion or prevention will take place [ 23 ]. As a result, considering the negative impact of sanctions, the financing system faces serious problems in three main functions of resource collection, pooling, and purchasing. The ability of countries to achieve health system’s goals largely depends on the knowledge, skills, motivation and deployment of individuals to organize and provide health services [ 23 ]. Numerous studies showed evidence of a direct relationship between health human resource and population outcomes [ 79 , 80 ].

Sanctions made it impossible to strengthen service delivery for achieving the Millennium Development Goals (MDGs) related to health by reducing the rate of occupied health workforce and forcing them to migrate. Studies assessed the impact of economic crises shown that these restrictions affected the health sector by increasing public vulnerability as same as the inability to meet public needs and expectations due to limited resources [ 81 , 82 , 83 ]. The findings from Iran show that sanctions can influence health care delivery adversely during health pandemics like COVID-19 disease in various direct and indirect ways [ 55 ]. Therefore, if sanctions continue, the reduction of inputs will hinder the improvement of service delivery and access to them. On the other hand, they disrupt access to health services even though with minimum quality standards.

About the research, sanctions limited and disqualified research activities by banning the access to most scientific and valid resources and disrupting international interactions and conferences. The small and isolated scientific communication has slowed down the health promotion progresses and limited the access to standards and protocols for promoting public health based on scientific evidence [ 31 , 45 ].

At the same time, economic sanctions affected on social determinants of health in various dimensions. Loss of markets, credits and favorable trading conditions reduces the value of the national currency and the ability to import goods. While, the cost of basic goods is strongly affected and prices increased [ 47 , 48 ]. Thus, sanctions forced severe negative effects on people’s health status by reduction of income, welfare, along with increasing unemployment and poverty [ 5 , 29 , 84 ]. This negative impact is more evident on the poor. These people cannot access or buy high or even sometimes low quality health care services.

Another effect of sanctions is reducing access to the food. Sanctions will restrict access to enough food for countries which import their agricultural products. Countries producing their own food products have better resilience. Limitation on access to basic material and food, as well as the economic pressures and declining incomes, affect the pattern of food consumption. Increasing the price of all kinds of food, turning to poor quality and unhealthy foods, as well as buying cheaper, low-nutrient ones, exacerbate malnutrition and make following a healthy diet impossible [ 69 , 70 , 85 ]. It should be noted that imposing sanctions are not always bad and sometimes they force countries to redesign their internal processes. The experience of Russian shows that although sanctions adversely influence agriculture to some extent but instead they played as the new momentum and help its dairy and milk sector to devise positive changes and increase the volume of inter-regional trade in milk and dairy products [ 86 ].

From a psychological point of view, the poor and fragile economic situation reduces the value of assets as same as the loss of purchasing power, which leads to increased frustration and stress in people. Frustration and despair cause and exacerbate various diseases [ 87 ]. Stressful situations also make people feel anxious, worried and unable to cope with. Psychosocial risks accumulate throughout life and increase the likelihood of poor mental health and premature death [ 23 ]. Findings from Iran proved that sanctions also affect mental health adversely. According to the WHO’ data, sanctions led to an increase in death due to self-harm and interpersonal violence in Iran (from 5.9 to 6.1 and from an average of 2.0 to 2.7 per 100,000 persons respectively) during the 2011–2014 period. Interestingly the self-harm related death reduced again in 2016, a year after lifting the sanctions [ 77 ].

Risky behaviors and addiction are taking as new behaviors patterns as a result of economic hardships and raised prices. Drug abuse has devastating effects on human health, while increase the rates of crime and mortality [ 88 ]. The highest incidence rate of HIV is among drug abusers and their sexual partners [ 89 ]. Thus, sanctions intensify these behaviors and patterns with a growing trend, which counts as a great threat to society and especially health system. A review of literature provides many evidence of sanctions effects on health while evidence was provided for most affected areas.

Many studies looked at the effects on access to medicines and medical equipment, research and health information systems, the socioeconomic situation, food and agriculture, and provided a clear picture of consequences. Although some others, focused on one specific area, the others discussed about the issues such as government or leadership, early life conditions, social isolation, social support, transportation which were less in number with no complete and clear evidence that needs further investigation. The most studies in our review examined the effects of sanctions using data and their analysis, or the pre- and post-sanctions situation. Therefore, according to our methodology and included studies, the extracted evidence is very valuable and reliable.

Understanding the impact of economic sanctions on health systems and social determinants of health is crucial for policymakers, highlighting the importance of collaborative global health governance. To address these challenges, a comprehensive approach is needed to minimize harm to vulnerable populations and promote a more equitable and resilient global health environment. Policymakers must reevaluate the effectiveness and unintended consequences of sanctions on health systems, prioritizing humanitarian concerns and ensuring that public health is not disproportionately affected. It is essential to explore alternative diplomatic strategies that allow for humanitarian exemptions within sanctions to guarantee the continued access to essential medical supplies for affected populations. Global health diplomacy should be leveraged to advocate for the removal or modification of sanctions hindering progress towards health-related Sustainable Development Goals (SDGs). Dialogue and negotiation should be prioritized to address underlying tensions while safeguarding the health and well-being of impacted communities. Establishing robust monitoring systems to track the impact of sanctions on health outcomes and social determinants is crucial. Strengthening multilateral collaborations and partnerships to address the health effects of sanctions is imperative, with international organizations like the World Health Organization (WHO) and United Nations (UN) playing a pivotal role in promoting global cooperation and finding solutions.

Strengths and limitations of the study

One of our limitations was the lack of access to the full text of some articles due to their publication time. Other studies may have been published in other languages about sanctions are excluded because of inclusion criteria. Other limitations may include losing articles about the impact of sanctions on various aspects of the health system that have lost their chance to be published due to political reasons.

Although the present study examined the impact of sanctions on the health system based on the Health System Building Blocks framework of the World Health Organization and using the approach of European SDH, it seems that in some areas the effects are not clear and further studies need to be done. However, given that the impact of sanctions varies from one country to another, the study has provided comprehensive evidence of the impacts along with consequences on health. The present evidence provides guide and helps with the adoption of international policies considering the goals of the WHO and the promotion of peace all around the world.

The results showed that economic sanctions imposed on different countries, directly and indirectly have strong negative impacts on health. Escalation of sanctions will be a severe threat and barrier for achieving the goal of global health coverage for everyone and everywhere. The international communities must work and focus on reducing the negative effects of these restrictions. They must anticipate the human effects and use whatever means are needed to prevent them. Some of these negative effects like disability and death, are irreversible. Therefore, it seems better for decision makers to recommend an international prescriptive to prevent such irreparable effects on the population of target countries before imposing sanctions.

Data availability

All data generated during the current study would be available from the corresponding author on reasonable request.

Abbreviations

Social Determinants of Health

Sustainable Development Goals

World Health Organization

Davis L, Engerman S. History lessons sanctions: neither war nor peace. J Economic Perspect. 2003;17(2):187–97.

Article   Google Scholar  

M E. Effect of Economic Security sanctions on the Organization United Nations in the enjoyment of human rights. Q J Foreign Policy. 2012;21(1).

Garfield R, Santana S. The impact of the economic crisis and the US embargo on health in Cuba. Am J Public Health. 1997;87(1):15–20.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Peksen D. Economic sanctions and human security: the public health effect of economic sanctions. Foreign Policy Anal. 2011;7(3):237–51.

Gibbons E, Garfield R. The impact of economic sanctions on health and human rights in Haiti, 1991–1994. Am J Public Health. 1999;89(10):1499–504.

Hassani M. Impact of sanctions on cancer care in Iran. Archives Bone Joint Surg. 2018;6(4):248.

Google Scholar  

Gorji A. Sanctions against Iran: the impact on health services. 2014.

Shen J, Oraka E. Complementary and alternative medicine (CAM) use among children with current asthma. Prev Med. 2012;54(1):27–31.

Article   PubMed   Google Scholar  

Adams R, Smith B, Ruffin R. Factors associated with hospital admissions and repeat emergency department visits for adults with asthma. Thorax. 2000;55(7):566.

Kheirandish M, Rashidian A, Kebriaeezade A, Cheraghali AM, Soleymani F. A review of pharmaceutical policies in response to economic crises and sanctions. J Res Pharm Pract. 2015;4(3):115.

Article   PubMed   PubMed Central   Google Scholar  

Vogler S, Zimmermann N, Leopold C, de Joncheere K. Pharmaceutical policies in European countries in response to the global financial crisis. South med Rev. 2011;4(2):69.

Ahmadi AM, Meskarpour_amiri M. The public health effects of economic sanctions as a global concern in 21th century: why the economic sanctions is a cruel strategy. J Health Policy Sustainable Health. 2015;2(1).

Lim S. The paradox of unsustainability in UN Sustainable Development Goals (SDGs): the North Korean case in the context of accountability and the fragile states under sanctions. J Peace Unification. 2021;11(1):55–78.

Kokabisaghi F, Miller AC, Bashar FR, Salesi M, Zarchi AA, Keramatfar A et al. Impact of United States political sanctions on international collaborations and research in Iran. BMJ Global Health. 2019;4(5).

Solar O, Irwin A. A conceptual framework for action on the social determinants of health. WHO Document Production Services; 2010.

Yazdi-Feyzabadi V, Amini-Rarani M, Delavari S. The health consequences of economic sanctions: call for health diplomacy and international collaboration. Arch Iran Med. 2020;23(pecial):S51.

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Dehkordi AH, Mazaheri E, Ibrahim HA, Dalvand S, Gheshlagh RG. How to write a systematic review: a narrative review. Int J Prev Med. 2021;12.

Moher D, Liberati A, Tetzlaff J, Altman DG, Group* P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372.

Manyazewal T. Using the World Health Organization health system building blocks through survey of healthcare professionals to determine the performance of public healthcare facilities. Archives Public Health. 2017;75(1):1–8.

Organization WH. Everybody’s business–strengthening health systems to improve health outcomes: WHO’s framework for action. 2007.

Wilkinson RG, Marmot M. Social determinants of health: the solid facts. World Health Organization; 2003.

Abbas WAK, Azar NG, Haddad LG, Umlauf MG. Preconception health status of Iraqi women after trade embargo. Public Health Nurs. 2008;25(4):295–303.

Ali H. Hepatitis B infection among Iraqi children: the impact of sanctions. EMHJ-Eastern Mediterranean Health Journal, 10 (1–2), 6–11, 2004. 2004.

Al-Ani ZR, Al-Hiali SJ, Al-Farraji HH. Secular trend of infant mortality rate during wars and sanctions in Western Iraq. Saudi Med J. 2011;32(12):1267–73.

PubMed   Google Scholar  

Daponte BO, Garfield R. The effect of economic sanctions on the mortality of Iraqi children prior to the 1991 Persian Gulf War. Am J Public Health. 2000;90(4):546.

Ali MM, Shah IH. Sanctions and childhood mortality in Iraq. Lancet. 2000;355(9218):1851–7.

Article   CAS   PubMed   Google Scholar  

Akunjee M, Ali A. Healthcare under sanctions in Iraq: an elective experience. Med Confl Survival. 2002;18(3):249–57.

Ascherio A, Chase R, Cote T, Dehaes G, Hoskins E, Laaouej J, et al. Effect of the Gulf War on infant and child mortality in Iraq. N Engl J Med. 1992;327(13):931–6.

Younis MS, Aswad AM. The impact of war and economic sanctions on the mental health system in Iraq from 1990 to 2003: a preliminary report. Intervention J Mental Health Psychosocial Support Confl Affected Areas. 2018;16(1):54–8.

Al-Taee IKS, Al-Shamaa I. Long term follow-up of renal transplant patients-A single center experience in Iraq. Saudi J Kidney Dis Transplantation. 2005;16(1):40–5.

Garfield R, Leu C-S. A multivariate method for estimating mortality rates among children under 5 years from health and social indicators in Iraq. Int J Epidemiol. 2000;29(3):510–5.

Dewachi O, Rizk A, Singh NV. (Dis) connectivities in wartime: the therapeutic geographies of Iraqi healthcare–seeking in Lebanon. Glob Public Health. 2018;13(3):288–97.

Awqati NA, Ali MM, Al-Ward NJ, Majeed FA, Salman K, Al-Alak M, et al. Causes and differentials of childhood mortality in Iraq. BMC Pediatr. 2009;9:1–9.

Ghiasi G, Rashidian A, Kebriaeezadeh A, Salamzadeh J. The impact of the sanctions made against Iran on availability to asthma medicines in Tehran. Iran J Pharm Research: IJPR. 2016;15(3):567.

PubMed   PubMed Central   Google Scholar  

Kheirandish M, Rashidian A, Bigdeli M. A news media analysis of economic sanction effects on access to medicine in Iran. J Res Pharm Pract. 2015;4(4):199.

Rad EH, Delavari S, Aeenparast A, Afkar A, Farzadi F, Maftoon F. Does economic instability affect healthcare provision? Evidence based on the urban family physician program in Iran. Korean J Family Med. 2017;38(5):296.

Karimi M, Haghpanah S. The effects of economic sanctions on disease specific clinical outcomes of patients with thalassemia and hemophilia in Iran. Health Policy. 2015;119(2):239–43.

Kheirandish M, Varahrami V, Kebriaeezade A, Cheraghali AM. Impact of economic sanctions on access to noncommunicable diseases medicines in the Islamic Republic of Iran. East Mediterr Health J. 2018;24(1):42–51.

Deilamizade A, Esmizade S. Economic sanctions against Iran, and drug use in Tehran, Iran: a 2013 pilot study. Subst Use Misuse. 2015;50(7):859–68.

Asadi-Pooya AA, Tavana B, Tavana B, Emami M. Drug adherence of patients with epilepsy in Iran: the effects of the international economic sanctions. Acta Neurol Belgica. 2016;116:151–5.

Asadi-Pooya AA, Azizimalamiri R, Badv RS, Yarali B, Asadollahi M, Homayoun M, et al. Impacts of the international economic sanctions on Iranian patients with epilepsy. Epilepsy Behav. 2019;95:166–8.

Hosseini S, Rashtchi V, Kamali K, Moghimi M. Epidemiology and outcome of 2,590 burned patients in Northwest Iran. Annals Burns fire Disasters. 2017;30(2):85.

CAS   Google Scholar  

Almasi K, Jamali Mahmouie H, Yousefi A. Analysis of Razi Vaccin and Serum Scientific Institute re-earchers’ viewpoint on impact of Foreign Sanctions on Scientific Communications and Research activities. Scientometrics Res J. 2016;2(1):27–42.

Setayesh S, Mackey TK. Addressing the impact of economic sanctions on Iranian drug shortages in the joint comprehensive plan of action: promoting access to medicines and health diplomacy. Globalization Health. 2016;12(1):1–14.

Garfield R, Devin J, Fausey J. The health impact of economic sanctions. Bull N Y Acad Med. 1995;72(2):454.

CAS   PubMed   PubMed Central   Google Scholar  

Sen K, Al-Faisal W, AlSaleh Y. Syria: effects of conflict and sanctions on public health. J Public Health. 2013;35(2):195–9.

Moret ES. Humanitarian impacts of economic sanctions on Iran and Syria. Eur Secur. 2015;24(1):120–40.

Sahraian MA, Moghadasi AN, Eskandarieh S. Economic sanctions against Iran as an important factor in threatening the health of patients with multiple sclerosis. Curr J Neurol. 2021;20(1):15.

Bastani P, Dehghan Z, Kashfi SM, Dorosti H, Mohammadpour M, Mehralian G. Challenge of politico-economic sanctions on pharmaceutical procurement in Iran: a qualitative study. Iran J Med Sci. 2022;47(2):152.

Nayeri K, López-Pardo CM. Economic crisis and access to care: Cuba’s health care system since the collapse of the Soviet Union. Int J Health Serv. 2005;35(4):797–816.

Allen SH, Lektzian DJ. Economic sanctions: a blunt instrument? J Peace Res. 2013;50(1):121–35.

Faraji Dizaji S, Ghadamgahi ZS. The impact of smart and non-smart sanctions on government health expenditures: evidence from developing resource-based countries. 2021.

Abdoli A. Iran, sanctions, and the COVID-19 crisis. J Med Econ. 2020;23(12):1461–5.

Kim Y. Economic sanctions and child HIV. Int J Health Plann Manag. 2019;34(2):693–700.

Whitehall J. Snake bites in north east Sri Lanka. Rural Remote Health. 2007;7(4):1–6.

Ameri A, Barzegartahamtan M, Ghavamnasiri M, Mohammadpour R, Dehghan H, Sebzari A, et al. Current and future challenges of radiation oncology in Iran: a report from the Iranian society of clinical oncology. Clin Oncol. 2018;30(4):262–8.

Article   CAS   Google Scholar  

Shahabi S, Fazlalizadeh H, Stedman J, Chuang L, Shariftabrizi A, Ram R. The impact of international economic sanctions on Iranian cancer healthcare. Health Policy. 2015;119(10):1309–18.

Bezuidenhout L, Karrar O, Lezaun J, Nobes A. Economic sanctions and academia: overlooked impact and long-term consequences. PLoS ONE. 2019;14(10):e0222669.

Ha LT, Nam PX. An investigation of relationship between global economic sanction and life expectancy: do financial and institutional system matter? Dev Stud Res. 2022;9(1):48–66.

Gutmann J, Neuenkirch M, Neumeier F. Sanctioned to death? The impact of economic sanctions on life expectancy and its gender gap. J Dev Stud. 2021;57(1):139–62.

Gharibi F, Imani A, Haghi M, Dalal K. Relationship between political-economic sanctions and catastrophic health costs in multiple sclerosis patients in Iran. Iran J War Public Health. 2022;14(4):455–64.

Vlajinac HD, Marinković J, Kocev NI, Adanja BJ, Pekmezović T, Sipetić S, et al. Infectious diseases mortality in central Serbia. J Epidemiol Commun Health. 1997;51(2):172.

Hedges TR III, Hirano M, Tucker K, Caballero B. Epidemic optic and peripheral neuropathy in Cuba: a unique geopolitical public health problem. Surv Ophthalmol. 1997;41(4):341–53.

Shahabi S, Teymourlouy AA, Shabaninejad H, Kamali M, Lankarani KB, Mojgani P. Physical rehabilitation in Iran after international sanctions: explored findings from a qualitative study. Globalization Health. 2020;16(1):1–10.

Afesorgbor SK, Mahadevan R. The impact of economic sanctions on income inequality of target states. World Dev. 2016;83:1–11.

De Vos P, García-Fariñas A, Álvarez‐Pérez A, Rodríguez‐Salvá A, Bonet‐Gorbea M, Van der Stuyft P. Public health services, an essential determinant of health during crisis. Lessons from Cuba, 1989–2000. Tropical Med Int Health. 2012;17(4):469–79.

Hejazi J, Emamgholipour S. The effects of the re-imposition of US sanctions on food security in Iran. Int J Health Policy Manage. 2022;11(5):651.

Reid BC, Psoter WJ, Gebrian B, Wang MQ. The effect of an international embargo on malnutrition and childhood mortality in rural Haiti. Int J Health Serv. 2007;37(3):501–13.

Lv Z, Xu T. Do economic sanctions affect protectionism? Evidence from agricultural support. Econ Politics. 2019;31(1):27–42.

Afesorgbor SK. 23. Sanctioned to starve? The impact of economic sanctions on food security in targeted states. Research Handbook on Economic Sanctions. 2021:438.

Fotourehchi Z, Are. UN and US economic sanctions a cause or cure for the environment: empirical evidence from Iran. Environ Dev Sustain. 2020;22:5483–501.

Mladenovich D, Langeggen I. The impact of war and economic sanction on the incidence of retinopathy of prematurity in Serbia. J Visual Impairment Blindness. 2009;103(3):162–72.

van der Werff SD, Polman K, Ponce MC, Twisk JW, Díaz RJ, Gorbea MB, et al. Childhood atopic diseases and early life circumstances: an ecological study in Cuba. PLoS ONE. 2012;7(6):e39892.

Organization WH. Health and the millennium development goals. World Health Organization; 2005.

Aloosh M, Salavati A, Aloosh A. Economic sanctions threaten population health: the case of Iran. Public Health. 2019;169:10–3.

Danaei G, Harirchi I, Sajadi HS, Yahyaei F, Majdzadeh R. The harsh effects of sanctions on Iranian health. Lancet. 2019;394(10197):468–9.

Anand S, Bärnighausen T. Health workers and vaccination coverage in developing countries: an econometric analysis. Lancet. 2007;369(9569):1277–85.

Speybroeck N, Kinfu Y, Dal Poz MR, Evans DB. Reassessing the relationship between human resources for health, intervention coverage and health outcomes. Geneva: World Health Organization. 2006:1–14.

Rechel B, Suhrcke M, Tsolova S, Suk JE, Desai M, McKee M, et al. Economic crisis and communicable disease control in Europe: a scoping study among national experts. Health Policy. 2011;103(2–3):168–75.

Karanikolos M, Mladovsky P, Cylus J, Thomson S, Basu S, Stuckler D, et al. Financial crisis, austerity, and health in Europe. Lancet. 2013;381(9874):1323–31.

Kastanioti C, Kontodimopoulos N, Stasinopoulos D, Kapetaneas N, Polyzos N. Public procurement of health technologies in Greece in an era of economic crisis. Health Policy. 2013;109(1):7–13.

Abhari B, Aleemran R, Aghajani H. The effect of sanctions on Iran’s health system using provincial data and spatial panel methods from 2009 to 2016. J Health Adm. 2020;23(1):58–73.

Gillespie S, van den Bold M. Agriculture, food systems, and nutrition: meeting the challenge. Global Challenges. 2017;1(3):1600002.

Krivko M, Smutka L. Trade sanctions and agriculture support in milk and dairy industry: case of Russia. Sustainability. 2020;12(24):10325.

Griggs S. Hope and mental health in young adult college students: an integrative review. J PsychoSoc Nurs Ment Health Serv. 2017;55(2):28–35.

Njuho P, Davids A. Extent and influence of recreational drug use on men and women aged 15 years and older in South Africa. Afr J Drug Alcohol Stud. 2010;9(1).

Ali F, Nooshin K, Kianoosh N, Hengameh S, Abbas G. Modelling of HIV/AIDS in Iran up to 2014. J AIDS HIV Res Vol. 2011;3(12):231–9.

Download references

Acknowledgements

We are thankful to Kerman University of medical Sciences for preparing the required fund to do the study.

This study was supported financially by Institute for Futures Studies in Health, affiliated with Vice-Chancellery for Research and Technology of Kerman University of Medical Sciences.

Author information

Authors and affiliations.

Health Services Management Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman City, Iran

Vahid Yazdi-Feyzabadi & Anahita Behzadi

Faculty of Management and Medical Information Sciences, Kerman University of Medical Sciences, Kerman City, Iran

Atefeh Zolfagharnasab

Soheila Naghavi

Research Center of Tropical and Infectious Diseases, Kerman University of medical sciences, Kerman, Iran

Maysam Yousefi

Faculty of Health, Health Management and Economics Department, Ilam University of Medical Sciences, Ilam City, Iran

Mohammad Bazyar

You can also search for this author in PubMed   Google Scholar

Contributions

VYF proposed the topic, VYF and MB designed the study; AZ did the search strategy and identified articles. MB and SN screened the articles and extracted and classified the data; AB and MY supervised and contributed in classifications of findings. VYF and AB supervised the whole process of study from literature review to data extraction. MB and AB prepared and finalized the manuscript. All authors read and approved the manuscript for submission.

Corresponding author

Correspondence to Mohammad Bazyar .

Ethics declarations

Ethics approval and consent to participate.

The registration number of proposal is 97001030 and this study was approved by the ethics committee of Kerman University of Medical Sciences with the code of ethics IR.KMU.REC.1397.564.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Yazdi-Feyzabadi, V., Zolfagharnasab, A., Naghavi, S. et al. Direct and indirect effects of economic sanctions on health: a systematic narrative literature review. BMC Public Health 24 , 2242 (2024). https://doi.org/10.1186/s12889-024-19750-w

Download citation

Received : 22 February 2024

Accepted : 09 August 2024

Published : 17 August 2024

DOI : https://doi.org/10.1186/s12889-024-19750-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Economic sanctions
• Social determinants of health
• Health system
• Systematic review

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

• Small Language Model
• Computer Vision
• Federated Learning
• Reinforcement Learning
• Natural Language Processing
• New Releases
• Open Source AI
• AI Webinars
• 🔥 Promotion/Partnership

The study identifies AI risk frameworks by searching academic databases, consulting experts, and tracking forward and backward citations. This process led to the creating of an AI Risk Database containing 777 risks from 43 documents. The risks were categorized using a “best-fit framework synthesis” method, resulting in two taxonomies: a Causal Taxonomy, which classifies risks by entity, intent, and timing, and a Domain Taxonomy, which groups risks into seven key areas. These taxonomies can filter and analyze specific AI risks, aiding policymakers, auditors, academics, and industry professionals.

The study conducted a comprehensive review, analyzing 17,288 articles, and selected 43 relevant documents focused on AI risks. The documents included peer-reviewed articles, preprints, conference papers, and reports predominantly published after 2020. The study revealed diverse definitions and frameworks for AI risks, emphasizing a need for more standardized approaches. Two taxonomies—Causal and Domain—were used to categorize the identified risks, highlighting issues related to AI system safety, socioeconomic impacts, and ethical concerns like privacy and discrimination. The findings offer valuable insights for policymakers, auditors, and researchers, providing a structured foundation for understanding and mitigating AI risks.

The study offers detailed resources, including a website and database, to help you understand and address AI-related risks. It provides a foundation for discussion, research, and policy development without advocating for the importance of specific risks. The AI Risk Database categorizes risks into high-level and mid-level taxonomies, aiding in targeted mitigation efforts. The repository is comprehensive and adaptable, designed to support ongoing research and debate. 

Check out the Paper and Details . All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on  Twitter and join our  Telegram Channel and  LinkedIn Gr oup . If you like our work, you will love our  newsletter..

Don’t Forget to join our  48k+ ML SubReddit

Find Upcoming AI Webinars here

Arcee AI Introduces Arcee Swarm: A Groundbreaking Mixture of Agents MoA Architecture Inspired by the Cooperative Intelligence Found in Nature Itself

Sana Hassan

Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.

• Enhancing Reinforcement Learning Explainability with Temporal Reward Decomposition
• EmBARDiment: An Implicit Attention Framework that Enhances AI Interaction Efficiency in Extended Reality Through Eye-Tracking and Contextual Memory Integration
• AI and Cybersecurity: Navigating Innovation, Resilience, and Global Collaborative Efforts
• Google AI Released the Imagen 3 Technical Paper: Showcasing In-Depth Details

RELATED ARTICLES MORE FROM AUTHOR

Can llms visualize graphics assessing symbolic program understanding in ai, harvard and google researchers developed a novel communication learning approach to enhance decision-making in noisy restless multi-arm bandits, atomagents: a multi-agent ai system to autonomously design metallic alloys, saphira ai: an ai platform that revolutionizes hardware safety compliance, eth zurich researchers introduce data-driven linearization ddl: a novel algorithm in systematic linearization for dynamical systems, arablegaleval: a multitask ai benchmark dataset for assessing the arabic legal knowledge of llms, harvard and google researchers developed a novel communication learning approach to enhance decision-making in..., eth zurich researchers introduce data-driven linearization ddl: a novel algorithm in systematic linearization for..., google deepmind researchers propose a dynamic visual memory for flexible image classification, understanding the 27 unique challenges in large language model development: an empirical study of..., the challenges of implementing retrieval augmented generation (rag) in production, meet decisional ai: an ai agent for financial analysts.

• AI Magazine
• Privacy & TC
• Cookie Policy
• 🐝 Partnership and Promotion

Learn How To Curate Your AI Data At Scale [Webinar]

Thank You 🙌

Privacy Overview

To read this content please select one of the options below:

Please note you do not have access to teaching notes, the current state of integrating equity, diversity and inclusion into knowledge mobilization: a systematic literature review.

Equality, Diversity and Inclusion

ISSN : 2040-7149

Article publication date: 15 August 2024

The purpose of this paper is to systematically review and analyze the academic literature on integrating equity, diversity, and inclusion (EDI) into knowledge mobilization (KMb).

Design/methodology/approach

This systematic literature review of the body of scholarly literature published on integrating EDI with KMb follows established methods and protocols proposed by Popay et al . (2006) and Page et al . (2021). Using a relevant keyword string, a search was conducted in ProQuest and SCOPUS to find peer-reviewed articles, which were then screened using predetermined inclusion and exclusion criteria. Finally, inductive and deductive analyses were conducted on the selected articles.

The findings suggest that most of the authors are based in the Global North, the majority of literature was published in the last two years, and that it is conceptual. This synthesis highlights five solution-oriented themes: acknowledging inherent bias, centering marginalized groups, promoting responsible knowledge mobilization, establishing partnerships, and advocating for transformational and systemic change. This study also identifies four broad barriers: inherent, unconscious, and implicit biases, a lack of evidence-based best practices, siloing of research and information, and a lack of institutional support and resources. Findings also highlight the value of further research into barriers, gaps and opportunities.

Originality/value

By studying the intersection of EDI and KMb, this contemporary synthesis of the state of the field presents opportunities for future research into gaps, barriers and potential solutions.

• Knowledge dissemination
• Knowledge sharing
• Inclusive engagement
• Co-production
• Co-creation
• Science communication
• Citizen science
• Participatory action research (PAR)

Cornelius-Hernandez, T. and Clarke, A. (2024), "The current state of integrating equity, diversity and inclusion into knowledge mobilization: a systematic literature review", Equality, Diversity and Inclusion , Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/EDI-04-2023-0134

Emerald Publishing Limited

Copyright © 2024, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Headwinds to Understanding Stress Response Physiology: A Systematic Review Reveals Mismatch between Real and Simulated Marine Heatwaves

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Harmony A. Martell
• For correspondence: [email protected]
• Info/History
• Preview PDF

Laboratory experiments have long been used to guide predictions of organismal stress in response to our rapidly changing climate. However, the ability to simulate real world conditions in the laboratory can be a major barrier to prediction accuracy, creating obstacles to efforts informing ecosystem conservation and management. Capitalizing on an extensive experimental literature of coral bleaching physiology, we performed a systematic review of the literature and assembled a database to identify the methods being used to measure coral bleaching in heating experiments and assess how closely heating experiments resembled marine heatwaves (MHWs) on coral reefs. Observations of the maximum photochemical yield of Photosystem II (FV/FM), though not a direct measure of bleaching, vastly outnumbered Symbiodiniaceae density and chlorophyll (ug cm-2, pg cell-1) observations in the available literature, indicating the widespread misuse of FV/FM as a proxy for coral bleaching. Laboratory studies in our database used significantly higher maximum temperatures, degree heating times (~ 1.7 x) and heating rates (~ 7.3 x), and significantly shorter durations (~ 1.5 x) than MHWs on coral reefs. We then asked whether exposure differences between lab and reef altered the relationship between coral bleaching and heating metrics using the example of hormesis, the biphasic dose response wherein low to moderate doses elicit some benefit, while high doses are deleterious. We fit curves on the data both with and without ecologically relevant heating metrics and found hormetic curves in some response variables were altered with the exclusion of exposures that fell outside of the bounds of MHWs on coral reefs. Differences between lab exposures and real-world MHWs were large enough to alter the relationships, indicating a high likelihood of prediction error. We recommend laboratory-based studies of coral bleaching use ecologically relevant exposures to improve our predictions of the coral physiological response to our rapidly warming oceans.

Competing Interest Statement

The authors have declared no competing interest.

View the discussion thread.

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Animal Behavior and Cognition (5526)
• Biochemistry (12575)
• Bioengineering (9444)
• Bioinformatics (30842)
• Biophysics (15864)
• Cancer Biology (12930)
• Cell Biology (18537)
• Clinical Trials (138)
• Developmental Biology (10005)
• Ecology (14979)
• Epidemiology (2067)
• Evolutionary Biology (19169)
• Genetics (12748)
• Genomics (17554)
• Immunology (12685)
• Microbiology (29741)
• Molecular Biology (12381)
• Neuroscience (64766)
• Paleontology (479)
• Pathology (2002)
• Pharmacology and Toxicology (3459)
• Physiology (5334)
• Plant Biology (11092)
• Scientific Communication and Education (1728)
• Synthetic Biology (3064)
• Systems Biology (7689)
• Zoology (1731)

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A systematic literature review of modalities, trends, and limitations in emotion recognition, affective computing, and sentiment analysis.

1. Introduction

2. methodology, 2.1. research questions, 2.2. search process, 2.2.1. search terms, 2.2.2. inclusion and exclusion criteria, 2.2.3. quality assessment, 2.2.4. data extraction, 3.1. overview, 3.2. unimodal data approaches, 3.2.1. unimodal physical approaches, 3.2.2. unimodal speech data approaches.

• Several articles mention the use of transfer learning for speech emotion recognition. This technique involves training models on one dataset and applying them to another. This can improve the efficiency of emotion recognition across different datasets.
• Some articles discuss multitask learning models, which are designed to simultaneously learn multiple related tasks. In the context of speech emotion recognition, this approach may help capture commonalities and differences across different datasets or emotions.
• Data augmentation techniques are mentioned in multiple articles, which involve generating additional training data from existing data, which can improve model performance and generalization.
• Attention mechanisms are a common trend for improving emotion recognition. Attention models allow the model to focus on specific features or segments of the input data that are most relevant for recognizing emotions, such as in multi-level attention-based approaches.
• Many articles discuss the use of deep learning models, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and some variants like “Two-Stage Fuzzy Fusion Based-Convolution Neural Network, “Deep Convolutional LSTM”, and “Attention-Oriented Parallel CNN Encoders”.
• While deep learning is prevalent, some articles explore novel feature engineering methods, such as modulation spectral features and wavelet packet information gain entropy, to enhance emotion recognition.
• From the list of articles on unimodal emotion recognition through speech, 7.14% address the challenge of recognizing emotions across different datasets or corpora. This is an important trend for making emotion recognition models more versatile.
• A few articles focus on making emotion recognition models more interpretable and explainable, which is crucial for real-world applications and understanding how the model makes its predictions.
• Ensemble methods, which combine multiple models to make predictions, are mentioned in several articles as a way to improve the performance of emotion recognition systems.
• Some articles discuss emotion recognition in specific contexts, such as call/contact centers, school violence detection, depression detection, analysis of podcast recordings, noisy environment analysis, in-the-wild sentiment analysis, and speech emotion segmentation of vowel-like and non-vowel-like regions. This indicates a trend toward applying emotion recognition in diverse applications.

3.2.3. Unimodal Text Data Approaches

3.2.4. unimodal physiological data approaches.

• Attention and self-attention mechanisms: These suggest that researchers are paying attention to the relevance of different parts of EEG signals for emotion recognition.
• Generative adversarial networks (GANs): Used for generating synthetic EEG data in order to improve the robustness and generalization of the models.
• Semi-supervised learning and domain transfer: Allow emotion recognition with limited datasets or datasets that are applicable to different domains, suggesting a concern for scalability and generalization of models.
• Interpretability and explainability: There is a growing interest in models that are interpretable and explainable, suggesting a concern for understanding how models make decisions and facilitating user trust in them.
• Utilization of transformers and capsule networks: Newer neural network architectures such as transformers and capsule networks are being explored for emotion recognition, indicating an interest in enhancing the modeling and representation capabilities of EEG signals.
• Although studies with a unimodal physical approach using signals different from EEG, like ECG, EDA, HR, and PPG, are still scarce, these can provide information about the cardiovascular system and the body’s autonomic response to emotions. Their limitations are that they may not be as specific or sensitive in detecting subtle or changing emotions. Noise and artifacts, such as motion, can affect the quality of these signals in practical situations and can be influenced by non-emotional factors, such as physical exercise and fatigue. Various studies explore the utilization of ECG and PPG signals for emotion recognition and stress classification. Techniques such as CNNs, LSTMs, attention mechanisms, self-supervised learning, and data augmentation are employed to analyze these signals and extract meaningful features for emotion recognition tasks. Bayesian deep learning frameworks are utilized for probabilistic modeling and uncertainty estimation in emotion prediction from HB data. These approaches aim to enhance human–computer interaction, improve mental health monitoring, and develop personalized systems for emotion recognition based on individual user characteristics.

3.3. Multi-Physical Data Approaches

• Most studies employ CNNs and RNNs, while others utilize variations of general neural networks, such as spiking neural networks (SNN) and tree-based neural networks. SNNs represent and transmit information through discrete bursts of neuronal activity, known as “spikes” or “pulses”, unlike conventional neural networks, which process information in continuous values. Additionally, several studies leverage advanced analysis models such as the stacked ensemble model and multimodal fusion models, which focus on integrating diverse sources of information to enhance decision-making. Transfer learning models and hybrid attention networks aim to capitalize on knowledge from related tasks or domains to improve performance in a target task. Attention-based neural networks prioritize capturing relevant information and patterns within the data. Semi-supervised and contrastive learning models offer alternative learning paradigms by incorporating both labeled and unlabeled data.
• The studies address diverse applications, including sarcasm, sentiment, and emotion recognition in conversations, financial distress prediction, performance evaluation in job interviews, emotion-based location recommendation systems, user experience (UX) analysis, emotion detection in video games, and in educational settings. This suggests that emotion recognition thorough multi-physical data analysis has a wide spectrum of applications in everyday life.
• Various audio and video signal processing techniques are employed, including pitch analysis, facial feature detection, cross-attention, and representational learning.

3.4. Multi-Physiological Data Approaches

• The fusion of physiological signals, such as EEG, ECG, PPG, GSR, EMG, BVP, EOG, respiration, temperature, and movement signals, is a predominant trend in these studies. The combination of multiple physiological signals allows for a richer representation of emotions.
• Most studies apply deep learning models, such as CNNs, RNNs, and autoencoder neural networks (AE), for the processing and analysis of these signals. Supervised and unsupervised learning approaches are also used.
• These studies focus on a variety of applications, such as emotion recognition in healthcare environments, brain–computer interfaces for music, emotion detection in interactive virtual environments, stress assessment in mobility environments for visually impaired people, among others. This indicates that emotion recognition based on physiological signals has applications in healthcare, technology, and beyond.
• Some studies focus on personalized emotion recognition, suggesting tailoring of models for each individual. This may be relevant for personalized health and wellness applications. Others focus on interactive applications and virtual environments useful for entertainment and virtual therapy.
• It is important to mention that the studies within this classification are quite limited in comparison to the previously described modalities. Although it appears that they are using similar physiological signals, the databases differ in terms of their approaches and generation methods. Therefore, there is an opportunity to establish a protocol for generating these databases, allowing for meaningful comparisons among studies.

3.5. Multi-Physical–Physiological Data Approaches

• Studies tend to combine multiple types of signals, such as EEG, facial expressions, voice signals, GSR, and other physiological data. Combining signals aims to take advantage of the complementarity of different modalities to improve accuracy in emotion detection.
• Machine learning models, in particular CNNs, are widely used in signal fusion for emotion recognition. CNN models can effectively process data from multiple modalities.
• Applications are also being explored in the health and wellness domain, such as emotion detection for emotional health analysis of people in smart environments.
• The use of standardized and widely accepted databases is important for comparing results between different studies; however, these are still limited.
• The trend towards non-intrusive sensors and wireless technology enables data collection in more natural and less intrusive environments, which facilitates the practical application of these systems in everyday environments.

4. Discussion

• Facial expression analysis approaches are currently being applied across various domains, including naturalistic settings (“in the wild”), on-road driver monitoring, virtual reality environments, smart homes, IoT and edge devices, and assistive robots. There is also a focus on mental health assessment, including autism, depression, and schizophrenia, and distinguishing between genuine and unfelt facial expressions of emotion. Efforts are being made to improve performance in processing faces acquired at a distance despite the challenges posed by low-quality images. Furthermore, there is an emerging interest in utilizing facial expression analysis in human–computer interaction (HCI), learning environments, and multicultural contexts.
• The recognition of emotions through speech and text has experienced tremendous growth, largely due to the abundance of information facilitated by advancements in technology and social media. This has enabled individuals to express their opinions and sentiments through various media, including podcast recordings, live videos, and readily available data sources such as social media platforms like Twitter, Facebook, Instagram, and blogs. Additionally, researchers have utilized unconventional sources like stock market data and tourism-related reviews. The variety and richness of these data sources indicate a wide range of segments where such emotion recognition analyses can be applied effectively.
• EEG signals continue to be a prominent modality for emotion recognition due to their highly accurate insights into emotional states. Between 2022 and 2023, studies in this field experienced exponential growth. The identified trends include utilizing EEG for enhancing human–computer interaction, recognizing emotions in various contexts such as patients with consciousness disorders, movie viewing, virtual environments, and driving scenarios. EEG is being used for detecting and monitoring mental health issues. There is also a growing focus on personalization, leading towards more individualized and user-specific emotion recognition systems, Other physiological signals, such as ECG, EDA, and HR, are also gaining attention, albeit at a slower pace.
• In the realm of multi-physical, multi-physiological, and multi-physical–physiological approaches, it is the former that appears to be laying the groundwork, as evidenced by the abundance of studies in this area. The latter two approaches, incorporating fusions with physiological signals, are still relatively scarce but seem to be paving the way for future researchers to contribute to their growth. Multimodal approaches, which integrate both physical and physiological signals, are finding diverse applications in emotion recognition. These range from healthcare systems, individual and group mood research, personality recognition, pain intensity recognition, anxiety detection, work stress detection, stress classification and security monitoring in public spaces, to vehicle security monitoring, movie audience emotion recognition, applications for autism spectrum disorder detection, music interfacing, and virtual environments.
• Bidirectional encoder representations from transformers: Used in sentiment analysis and emotion recognition from text, BERT models can understand the context of words in sentences by pre-training on a large text and then fine-tuning for specific tasks like sentiment analysis.
• CNNs: These are commonly applied in facial emotion recognition, emotion recognition from physiological signals, and even in speech emotion recognition by analyzing spectrograms.
• RNNS and variants (LSTM, GRU): These models are suited for sequential data like speech and text. LSTMs and GRUs are particularly effective in speech emotion recognition and sentiment analysis of time-series data.
• Graph convolutional networks (GCNs): Applied in emotion recognition from EEG signals and conversation-based emotion recognition, these can model relational data and capture the complex dependencies in graph-structured data, like brain connectivity patterns or conversational contexts.
• Attention mechanisms and transformers: Enhancing the ability of models to focus on relevant parts of the data, attention mechanisms are integral to models like transformers for tasks that require understanding the context, such as sentiment analysis in long documents or emotion recognition in conversations.
• Ensemble models: Combining predictions from multiple models to improve accuracy, ensemble methods are used in multimodal emotion recognition, where inputs from different modalities (e.g., audio, text, and video) are integrated to make more accurate predictions.
• Autoencoders and generative adversarial networks (GANs): For tasks like data augmentation in emotion recognition from EEG or for generating synthetic data to improve model robustness, these unsupervised learning models can learn compact representations of data or generate new data samples, respectively.
• Multimodal fusion models: In applications requiring the integration of multiple data types (e.g., speech, text, and video for emotion recognition), fusion models combine features from different modalities to capture more comprehensive information for prediction tasks.
• Transfer learning: Utilizing pre-trained models on large datasets and fine-tuning them for specific affective computing tasks, transfer learning is particularly useful in scenarios with limited labeled data, such as sentiment analysis in niche domains.
• Spatio-temporal models: For tasks that involve data with both spatial and temporal dimensions (like video-based emotion recognition or physiological signal analysis), models that capture spatio-temporal dynamics are employed, combining approaches like CNNs for spatial features and RNNs/LSTMs for temporal features.

5. Conclusions

Author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Zhou, T.H.; Liang, W.; Liu, H.; Wang, L.; Ryu, K.H.; Nam, K.W. EEG Emotion Recognition Applied to the Effect Analysis of Music on Emotion Changes in Psychological Healthcare. Int. J. Environ. Res. Public Health 2022 , 20 , 378. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hajek, P.; Munk, M. Speech Emotion Recognition and Text Sentiment Analysis for Financial Distress Prediction. Neural Comput. Appl. 2023 , 35 , 21463–21477. [ Google Scholar ] [ CrossRef ]
• Naim, I.; Tanveer, M.d.I.; Gildea, D.; Hoque, M.E. Automated Analysis and Prediction of Job Interview Performance. IEEE Trans. Affect. Comput. 2018 , 9 , 191–204. [ Google Scholar ] [ CrossRef ]
• Ayata, D.; Yaslan, Y.; Kamasak, M.E. Emotion Recognition from Multimodal Physiological Signals for Emotion Aware Healthcare Systems. J. Med. Biol. Eng. 2020 , 40 , 149–157. [ Google Scholar ] [ CrossRef ]
• Maithri, M.; Raghavendra, U.; Gudigar, A.; Samanth, J.; Barua, D.P.; Murugappan, M.; Chakole, Y.; Acharya, U.R. Automated Emotion Recognition: Current Trends and Future Perspectives. Comput. Methods Programs Biomed. 2022 , 215 , 106646. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Du, Z.; Wu, S.; Huang, D.; Li, W.; Wang, Y. Spatio-Temporal Encoder-Decoder Fully Convolutional Network for Video-Based Dimensional Emotion Recognition. IEEE Trans. Affect. Comput. 2021 , 12 , 565–578. [ Google Scholar ] [ CrossRef ]
• Montero Quispe, K.G.; Utyiama, D.M.S.; dos Santos, E.M.; Oliveira, H.A.B.F.; Souto, E.J.P. Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals. Sensors 2022 , 22 , 9102. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Wang, J.; Liu, Y.; Rong, L.; Zheng, Q.; Song, D.; Tiwari, P.; Qin, J. A Multitask Learning Model for Multimodal Sarcasm, Sentiment and Emotion Recognition in Conversations. Inf. Fusion 2023 , 93 , 282–301. [ Google Scholar ] [ CrossRef ]
• Leong, S.C.; Tang, Y.M.; Lai, C.H.; Lee, C.K.M. Facial Expression and Body Gesture Emotion Recognition: A Systematic Review on the Use of Visual Data in Affective Computing. Comput. Sci. Rev. 2023 , 48 , 100545. [ Google Scholar ] [ CrossRef ]
• Aranha, R.V.; Correa, C.G.; Nunes, F.L.S. Adapting Software with Affective Computing: A Systematic Review. IEEE Trans. Affect. Comput. 2021 , 12 , 883–899. [ Google Scholar ] [ CrossRef ]
• Kratzwald, B.; Ilić, S.; Kraus, M.; Feuerriegel, S.; Prendinger, H. Deep Learning for Affective Computing: Text-Based Emotion Recognition in Decision Support. Decis. Support. Syst. 2018 , 115 , 24–35. [ Google Scholar ] [ CrossRef ]
• Ab. Aziz, N.A.; K., T.; Ismail, S.N.M.S.; Hasnul, M.A.; Ab. Aziz, K.; Ibrahim, S.Z.; Abd. Aziz, A.; Raja, J.E. Asian Affective and Emotional State (A2ES) Dataset of ECG and PPG for Affective Computing Research. Algorithms 2023 , 16 , 130. [ Google Scholar ] [ CrossRef ]
• Jung, T.-P.; Sejnowski, T.J. Utilizing Deep Learning Towards Multi-Modal Bio-Sensing and Vision-Based Affective Computing. IEEE Trans. Affect. Comput. 2022 , 13 , 96–107. [ Google Scholar ] [ CrossRef ]
• Shah, S.; Ghomeshi, H.; Vakaj, E.; Cooper, E.; Mohammad, R. An Ensemble-Learning-Based Technique for Bimodal Sentiment Analysis. Big Data Cogn. Comput. 2023 , 7 , 85. [ Google Scholar ] [ CrossRef ]
• Tang, J.; Hou, M.; Jin, X.; Zhang, J.; Zhao, Q.; Kong, W. Tree-Based Mix-Order Polynomial Fusion Network for Multimodal Sentiment Analysis. Systems 2023 , 11 , 44. [ Google Scholar ] [ CrossRef ]
• Khamphakdee, N.; Seresangtakul, P. An Efficient Deep Learning for Thai Sentiment Analysis. Data 2023 , 8 , 90. [ Google Scholar ] [ CrossRef ]
• Jo, A.-H.; Kwak, K.-C. Speech Emotion Recognition Based on Two-Stream Deep Learning Model Using Korean Audio Information. Appl. Sci. 2023 , 13 , 2167. [ Google Scholar ] [ CrossRef ]
• Abdulrahman, A.; Baykara, M.; Alakus, T.B. A Novel Approach for Emotion Recognition Based on EEG Signal Using Deep Learning. Appl. Sci. 2022 , 12 , 10028. [ Google Scholar ] [ CrossRef ]
• Middya, A.I.; Nag, B.; Roy, S. Deep Learning Based Multimodal Emotion Recognition Using Model-Level Fusion of Audio–Visual Modalities. Knowl. Based Syst. 2022 , 244 , 108580. [ Google Scholar ] [ CrossRef ]
• Ali, M.; Mosa, A.H.; Al Machot, F.; Kyamakya, K. EEG-Based Emotion Recognition Approach for e-Healthcare Applications. In Proceedings of the 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), Vienna, Austria, 5–8 July 2016; pp. 946–950. [ Google Scholar ]
• Zepf, S.; Hernandez, J.; Schmitt, A.; Minker, W.; Picard, R.W. Driver Emotion Recognition for Intelligent Vehicles. ACM Comput. Surv. (CSUR) 2020 , 53 , 1–30. [ Google Scholar ] [ CrossRef ]
• Zaman, K.; Zhaoyun, S.; Shah, B.; Hussain, T.; Shah, S.M.; Ali, F.; Khan, U.S. A Novel Driver Emotion Recognition System Based on Deep Ensemble Classification. Complex. Intell. Syst. 2023 , 9 , 6927–6952. [ Google Scholar ] [ CrossRef ]
• Du, Y.; Crespo, R.G.; Martínez, O.S. Human Emotion Recognition for Enhanced Performance Evaluation in E-Learning. Prog. Artif. Intell. 2022 , 12 , 199–211. [ Google Scholar ] [ CrossRef ]
• Alaei, A.; Wang, Y.; Bui, V.; Stantic, B. Target-Oriented Data Annotation for Emotion and Sentiment Analysis in Tourism Related Social Media Data. Future Internet 2023 , 15 , 150. [ Google Scholar ] [ CrossRef ]
• Caratù, M.; Brescia, V.; Pigliautile, I.; Biancone, P. Assessing Energy Communities’ Awareness on Social Media with a Content and Sentiment Analysis. Sustainability 2023 , 15 , 6976. [ Google Scholar ] [ CrossRef ]
• Bota, P.J.; Wang, C.; Fred, A.L.N.; Placido Da Silva, H. A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals. IEEE Access 2019 , 7 , 140990–141020. [ Google Scholar ] [ CrossRef ]
• Egger, M.; Ley, M.; Hanke, S. Emotion Recognition from Physiological Signal Analysis: A Review. Electron. Notes Theor. Comput. Sci. 2019 , 343 , 35–55. [ Google Scholar ] [ CrossRef ]
• Shu, L.; Xie, J.; Yang, M.; Li, Z.; Li, Z.; Liao, D.; Xu, X.; Yang, X. A Review of Emotion Recognition Using Physiological Signals. Sensors 2018 , 18 , 2074. [ Google Scholar ] [ CrossRef ]
• Canal, F.Z.; Müller, T.R.; Matias, J.C.; Scotton, G.G.; de Sa Junior, A.R.; Pozzebon, E.; Sobieranski, A.C. A Survey on Facial Emotion Recognition Techniques: A State-of-the-Art Literature Review. Inf. Sci. 2022 , 582 , 593–617. [ Google Scholar ] [ CrossRef ]
• Assabumrungrat, R.; Sangnark, S.; Charoenpattarawut, T.; Polpakdee, W.; Sudhawiyangkul, T.; Boonchieng, E.; Wilaiprasitporn, T. Ubiquitous Affective Computing: A Review. IEEE Sens. J. 2022 , 22 , 1867–1881. [ Google Scholar ] [ CrossRef ]
• Schmidt, P.; Reiss, A.; Dürichen, R.; Laerhoven, K. Van Wearable-Based Affect Recognition—A Review. Sensors 2019 , 19 , 4079. [ Google Scholar ] [ CrossRef ]
• Rouast, P.V.; Adam, M.T.P.; Chiong, R. Deep Learning for Human Affect Recognition: Insights and New Developments. IEEE Trans. Affect. Comput. 2021 , 12 , 524–543. [ Google Scholar ] [ CrossRef ]
• Ahmed, N.; Aghbari, Z.A.; Girija, S. A Systematic Survey on Multimodal Emotion Recognition Using Learning Algorithms. Intell. Syst. Appl. 2023 , 17 , 200171. [ Google Scholar ] [ CrossRef ]
• Kitchenham, B. Procedures for Performing Systematic Reviews ; Keele University: Keele, UK, 2004; Volume 33, pp. 1–26. [ Google Scholar ]
• Mollahosseini, A.; Hasani, B.; Mahoor, M.H. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Trans. Affect. Comput. 2019 , 10 , 18–31. [ Google Scholar ] [ CrossRef ]
• Al Jazaery, M.; Guo, G. Video-Based Depression Level Analysis by Encoding Deep Spatiotemporal Features. IEEE Trans. Affect. Comput. 2021 , 12 , 262–268. [ Google Scholar ] [ CrossRef ]
• Kollias, D.; Zafeiriou, S. Exploiting Multi-CNN Features in CNN-RNN Based Dimensional Emotion Recognition on the OMG in-the-Wild Dataset. IEEE Trans. Affect. Comput. 2021 , 12 , 595–606. [ Google Scholar ] [ CrossRef ]
• Li, S.; Deng, W. A Deeper Look at Facial Expression Dataset Bias. IEEE Trans. Affect. Comput. 2022 , 13 , 881–893. [ Google Scholar ] [ CrossRef ]
• Kulkarni, K.; Corneanu, C.A.; Ofodile, I.; Escalera, S.; Baro, X.; Hyniewska, S.; Allik, J.; Anbarjafari, G. Automatic Recognition of Facial Displays of Unfelt Emotions. IEEE Trans. Affect. Comput. 2021 , 12 , 377–390. [ Google Scholar ] [ CrossRef ]
• Punuri, S.B.; Kuanar, S.K.; Kolhar, M.; Mishra, T.K.; Alameen, A.; Mohapatra, H.; Mishra, S.R. Efficient Net-XGBoost: An Implementation for Facial Emotion Recognition Using Transfer Learning. Mathematics 2023 , 11 , 776. [ Google Scholar ] [ CrossRef ]
• Mukhiddinov, M.; Djuraev, O.; Akhmedov, F.; Mukhamadiyev, A.; Cho, J. Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People. Sensors 2023 , 23 , 1080. [ Google Scholar ] [ CrossRef ]
• Babu, E.K.; Mistry, K.; Anwar, M.N.; Zhang, L. Facial Feature Extraction Using a Symmetric Inline Matrix-LBP Variant for Emotion Recognition. Sensors 2022 , 22 , 8635. [ Google Scholar ] [ CrossRef ]
• Mustafa Hilal, A.; Elkamchouchi, D.H.; Alotaibi, S.S.; Maray, M.; Othman, M.; Abdelmageed, A.A.; Zamani, A.S.; Eldesouki, M.I. Manta Ray Foraging Optimization with Transfer Learning Driven Facial Emotion Recognition. Sustainability 2022 , 14 , 14308. [ Google Scholar ] [ CrossRef ]
• Bisogni, C.; Cimmino, L.; De Marsico, M.; Hao, F.; Narducci, F. Emotion Recognition at a Distance: The Robustness of Machine Learning Based on Hand-Crafted Facial Features vs Deep Learning Models. Image Vis. Comput. 2023 , 136 , 104724. [ Google Scholar ] [ CrossRef ]
• Sun, Q.; Liang, L.; Dang, X.; Chen, Y. Deep Learning-Based Dimensional Emotion Recognition Combining the Attention Mechanism and Global Second-Order Feature Representations. Comput. Electr. Eng. 2022 , 104 , 108469. [ Google Scholar ] [ CrossRef ]
• Sudha, S.S.; Suganya, S.S. On-Road Driver Facial Expression Emotion Recognition with Parallel Multi-Verse Optimizer (PMVO) and Optical Flow Reconstruction for Partial Occlusion in Internet of Things (IoT). Meas. Sens. 2023 , 26 , 100711. [ Google Scholar ] [ CrossRef ]
• Barra, P.; De Maio, L.; Barra, S. Emotion Recognition by Web-Shaped Model. Multimed. Tools Appl. 2023 , 82 , 11321–11336. [ Google Scholar ] [ CrossRef ]
• Bhattacharya, A.; Choudhury, D.; Dey, D. Edge-Enhanced Bi-Dimensional Empirical Mode Decomposition-Based Emotion Recognition Using Fusion of Feature Set. Soft Comput. 2018 , 22 , 889–903. [ Google Scholar ] [ CrossRef ]
• Lucey, P.; Cohn, J.F.; Kanade, T.; Saragih, J.; Ambadar, Z.; Matthews, I. The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit and Emotion-Specified Expression. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA, 13–18 June 2010; pp. 94–101. [ Google Scholar ]
• Zhao, G.; Huang, X.; Taini, M.; Li, S.Z.; Pietikäinen, M. Facial Expression Recognition from Near-Infrared Videos. Image Vis. Comput. 2011 , 29 , 607–619. [ Google Scholar ] [ CrossRef ]
• Barros, P.; Churamani, N.; Lakomkin, E.; Siqueira, H.; Sutherland, A.; Wermter, S. The OMG-Emotion Behavior Dataset. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–7. [ Google Scholar ]
• Ullah, Z.; Qi, L.; Hasan, A.; Asim, M. Improved Deep CNN-Based Two Stream Super Resolution and Hybrid Deep Model-Based Facial Emotion Recognition. Eng. Appl. Artif. Intell. 2022 , 116 , 105486. [ Google Scholar ] [ CrossRef ]
• Zheng, W.; Zong, Y.; Zhou, X.; Xin, M. Cross-Domain Color Facial Expression Recognition Using Transductive Transfer Subspace Learning. IEEE Trans. Affect. Comput. 2018 , 9 , 21–37. [ Google Scholar ] [ CrossRef ]
• Tan, K.L.; Lee, C.P.; Lim, K.M. RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis. Appl. Sci. 2023 , 13 , 3915. [ Google Scholar ] [ CrossRef ]
• Ren, M.; Huang, X.; Li, W.; Liu, J. Multi-Loop Graph Convolutional Network for Multimodal Conversational Emotion Recognition. J. Vis. Commun. Image Represent. 2023 , 94 , 103846. [ Google Scholar ] [ CrossRef ]
• Mai, S.; Hu, H.; Xu, J.; Xing, S. Multi-Fusion Residual Memory Network for Multimodal Human Sentiment Comprehension. IEEE Trans. Affect. Comput. 2022 , 13 , 320–334. [ Google Scholar ] [ CrossRef ]
• Yang, L.; Jiang, D.; Sahli, H. Integrating Deep and Shallow Models for Multi-Modal Depression Analysis—Hybrid Architectures. IEEE Trans. Affect. Comput. 2021 , 12 , 239–253. [ Google Scholar ] [ CrossRef ]
• Mocanu, B.; Tapu, R.; Zaharia, T. Multimodal Emotion Recognition Using Cross Modal Audio-Video Fusion with Attention and Deep Metric Learning. Image Vis. Comput. 2023 , 133 , 104676. [ Google Scholar ] [ CrossRef ]
• Noroozi, F.; Marjanovic, M.; Njegus, A.; Escalera, S.; Anbarjafari, G. Audio-Visual Emotion Recognition in Video Clips. IEEE Trans. Affect. Comput. 2019 , 10 , 60–75. [ Google Scholar ] [ CrossRef ]
• Davison, A.K.; Lansley, C.; Costen, N.; Tan, K.; Yap, M.H. SAMM: A Spontaneous Micro-Facial Movement Dataset. IEEE Trans. Affect. Comput. 2018 , 9 , 116–129. [ Google Scholar ] [ CrossRef ]
• Happy, S.L.; Routray, A. Fuzzy Histogram of Optical Flow Orientations for Micro-Expression Recognition. IEEE Trans. Affect. Comput. 2019 , 10 , 394–406. [ Google Scholar ] [ CrossRef ]
• Schmidt, P.; Reiss, A.; Duerichen, R.; Marberger, C.; Van Laerhoven, K. Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection. In Proceedings of the Proceedings of the 20th ACM International Conference on Multimodal Interaction, New York, NY, USA, 2 October 2018; ACM: New York, NY, USA, 2018; pp. 400–408. [ Google Scholar ]
• Miranda-Correa, J.A.; Abadi, M.K.; Sebe, N.; Patras, I. AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups. IEEE Trans. Affect. Comput. 2021 , 12 , 479–493. [ Google Scholar ] [ CrossRef ]
• Subramanian, R.; Wache, J.; Abadi, M.K.; Vieriu, R.L.; Winkler, S.; Sebe, N. ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors. IEEE Trans. Affect. Comput. 2018 , 9 , 147–160. [ Google Scholar ] [ CrossRef ]
• Koelstra, S.; Muhl, C.; Soleymani, M.; Lee, J.-S.; Yazdani, A.; Ebrahimi, T.; Pun, T.; Nijholt, A.; Patras, I. DEAP: A Database for Emotion Analysis; Using Physiological Signals. IEEE Trans. Affect. Comput. 2012 , 3 , 18–31. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Cheng, C.; Wang, S.; Xia, T. Emotion Recognition Using Heterogeneous Convolutional Neural Networks Combined with Multimodal Factorized Bilinear Pooling. Biomed. Signal Process Control 2022 , 77 , 103877. [ Google Scholar ] [ CrossRef ]
• The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. Available online: https://www.prisma-statement.org/prisma-2020-statement (accessed on 12 August 2024).

Click here to enlarge figure

Share and Cite

García-Hernández, R.A.; Luna-García, H.; Celaya-Padilla, J.M.; García-Hernández, A.; Reveles-Gómez, L.C.; Flores-Chaires, L.A.; Delgado-Contreras, J.R.; Rondon, D.; Villalba-Condori, K.O. A Systematic Literature Review of Modalities, Trends, and Limitations in Emotion Recognition, Affective Computing, and Sentiment Analysis. Appl. Sci. 2024 , 14 , 7165. https://doi.org/10.3390/app14167165

García-Hernández RA, Luna-García H, Celaya-Padilla JM, García-Hernández A, Reveles-Gómez LC, Flores-Chaires LA, Delgado-Contreras JR, Rondon D, Villalba-Condori KO. A Systematic Literature Review of Modalities, Trends, and Limitations in Emotion Recognition, Affective Computing, and Sentiment Analysis. Applied Sciences . 2024; 14(16):7165. https://doi.org/10.3390/app14167165

García-Hernández, Rosa A., Huizilopoztli Luna-García, José M. Celaya-Padilla, Alejandra García-Hernández, Luis C. Reveles-Gómez, Luis Alberto Flores-Chaires, J. Ruben Delgado-Contreras, David Rondon, and Klinge O. Villalba-Condori. 2024. "A Systematic Literature Review of Modalities, Trends, and Limitations in Emotion Recognition, Affective Computing, and Sentiment Analysis" Applied Sciences 14, no. 16: 7165. https://doi.org/10.3390/app14167165

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals