research paper about the impact of covid 19

Advertisement

The Impact of COVID-19 on Education: A Meta-Narrative Review

• Original Paper
• Published: 05 July 2022
• Volume 66 , pages 883–896, ( 2022 )

Cite this article

• Aras Bozkurt   ORCID: orcid.org/0000-0002-4520-642X 1 , 2 , 3 ,
• Kadir Karakaya   ORCID: orcid.org/0000-0003-3375-1532 4 ,
• Murat Turk   ORCID: orcid.org/0000-0002-5105-2578 5 ,
• Özlem Karakaya   ORCID: orcid.org/0000-0002-9950-481X 6 &
• Daniela Castellanos-Reyes   ORCID: orcid.org/0000-0002-0183-1549 7  

15k Accesses

46 Citations

52 Altmetric

Explore all metrics

The rapid and unexpected onset of the COVID-19 global pandemic has generated a great degree of uncertainty about the future of education and has required teachers and students alike to adapt to a new normal to survive in the new educational ecology. Through this experience of the new educational ecology, educators have learned many lessons, including how to navigate through uncertainty by recognizing their strengths and vulnerabilities. In this context, the aim of this study is to conduct a bibliometric analysis of the publications covering COVID-19 and education to analyze the impact of the pandemic by applying the data mining and analytics techniques of social network analysis and text-mining. From the abstract, title, and keyword analysis of a total of 1150 publications, seven themes were identified: (1) the great reset, (2) shifting educational landscape and emerging educational roles (3) digital pedagogy, (4) emergency remote education, (5) pedagogy of care, (6) social equity, equality, and injustice, and (7) future of education. Moreover, from the citation analysis, two thematic clusters emerged: (1) educational response, emergency remote education affordances, and continuity of education, and (2) psychological impact of COVID-19. The overlap between themes and thematic clusters revealed researchers’ emphasis on guaranteeing continuity of education and supporting the socio-emotional needs of learners. From the results of the study, it is clear that there is a heightened need to develop effective strategies to ensure the continuity of education in the future, and that it is critical to proactively respond to such crises through resilience and flexibility.

Similar content being viewed by others

The Impact of COVID-19 on Higher Education: A Systematic Literature Review of Pedagogical Approaches and Challenges

Learning from a Pandemic. The Impact of COVID-19 on Education Around the World

Effects of COVID-19 Pandemic on Higher Education. A Bibliometric Perspective

Explore related subjects.

• Artificial Intelligence
• Digital Education and Educational Technology

Avoid common mistakes on your manuscript.

Introduction

The Coronavirus (COVID-19) pandemic has proven to be a massive challenge for the entire world, imposing a radical transformation in many areas of life, including education. It was rapid and unexpected; the world was unprepared and hit hard. The virus is highly contagious, having a pathogenic nature whose effects have not been limited to humans alone, but rather, includes every construct and domain of societies, including education. The education system, which has been affected at all levels, has been required to respond to the crisis, forced to transition into emergency modes, and adapt to the unprecedented impact of the global crisis. Although the beginning of 2021 will mark nearly a year of experience in living through the pandemic, the crisis remains a phenomenon with many unknowns. A deeper and more comprehensive understanding of the changes that have been made in response to the crisis is needed to survive in these hard times. Hence, this study aims to provide a better understanding by examining the scholarly publications on COVID-19 and education. In doing this, we can identify our weaknesses and vulnerabilities, be better prepared for the new normal, and be more fit to survive.

Related Literature

Though the COVID-19 pandemic is not the first major disruption to be experienced in the history of the world, it has been unique due to its scale and the requirements that have been imposed because of it (Guitton, 2020 ). The economies of many countries have greatly suffered from the lockdowns and other restrictive measurements, and people have had to adapt to a new lifestyle, where their primary concern is to survive by keeping themselves safe from contracting the deadly virus. The education system has not been exempt from this series of unfortunate events inflicted by COVID-19. Since brick-and-mortar schools had to be closed due to the pandemic, millions of students, from those in K-12 to those in higher education, were deprived of physical access to their classrooms, peers, and teachers (Bozkurt & Sharma, 2020a , b ). This extraordinary pandemic period has posed arguably the most challenging and complex problems ever for educators, students, schools, educational institutions, parents, governments, and all other educational stakeholders. The closing of brick-and-mortar schools and campuses rendered online teaching and learning the only viable solution to the problem of access-to-education during this emergency period (Hodges et al., 2020 ). Due to the urgency of this move, teachers and instructors were rushed to shift all their face-to-face instruction and instructional materials to online spaces, such as learning management systems or electronic platforms, in order to facilitate teaching virtually at a distance. As a result of this sudden migration to learning and instruction online, the key distinctions between online education and education delivered online during such crisis and emergency circumstances have been obfuscated (Hodges et al., 2020 ).

State of the Current Relevant Literature

Although the scale of the impact of the COVID-19 global pandemic on education overshadows previously experienced nationwide or global crises or disruptions, the phenomenon of schools and higher education institutions having to shift their instruction to online spaces is not totally new to the education community and academia (Johnson et al., 2020 ). Prior literature on this subject indicates that in the past, schools and institutions resorted to online or electronic delivery of instruction in times of serious crises and uncertainties, including but not limited to natural disasters such as floods or earthquakes (e.g., Ayebi-Arthur, 2017 ; Lorenzo, 2008 ; Tull et al., 2017 ), local disruptions such as civil wars and socio-economic events such as political upheavals, social turmoils or economic recessions (e.g., Czerniewicz et al., 2019 ). Nevertheless, the past attempts to move learning and teaching online do not compare to the current efforts that have been implemented during the global COVID-19 pandemic, insofar as the past crisis situations were sporadic events in specific territories, affecting a limited population for relatively short periods of time. In contrast, the COVID-19 pandemic has continued to pose a serious threat to the continuity of education around the globe (Johnson et al., 2020 ).

Considering the scale and severity of the global pandemic, the impacts it has had on education in general and higher education in particular need to be explored and studied empirically so that necessary plans and strategies aimed at reducing its devastating effects can be developed and implemented. Due to the rapid onset and spread of the global pandemic, the current literature on the impact of COVID-19 on education is still limited, including mostly non-academic editorials or non-empirical personal reflections, anecdotes, reports, and stories (e.g., Baker, 2020 ; DePietro, 2020 ). Yet, with that said, empirical research on the impact of the global pandemic on higher education is rapidly growing. For example, Johnson et al. ( 2020 ), in their empirical study, found that faculty members who were struggling with various challenges adopted new instructional methods and strategies and adjusted certain course components to foster emergency remote education (ERE). Unger and Meiran ( 2020 ) observed that the pandemic made students in the US feel anxious about completing online learning tasks. In contrast, Suleri ( 2020 ) reported that a large majority of European higher education students were satisfied with their virtual learning experiences during the pandemic, and that most were willing to continue virtual higher education even after the pandemic (Suleri, 2020 ). The limited empirical research also points to the need for systematically planning and designing online learning experiences in advance in preparation for future outbreaks of such global pandemics and other crises (e.g., Korkmaz & Toraman, 2020 ). Despite the growing literature, the studies provide only fragmentary evidence on the impact of the pandemic on online learning and teaching. For a more thorough understanding of the serious implications the pandemic has for higher education in relation to learning and teaching online, more empirical research is needed.

Unlike previously conducted bibliometric analysis studies on this subject, which have largely involved general analysis of research on health sciences and COVID-19, Aristovnik et al. ( 2020 ) performed an in-depth bibliometric analysis of various science and social science research disciplines by examining a comprehensive database of document and source information. By the final phase of their bibliometric analysis, the authors had analyzed 16,866 documents. They utilized a mix of innovative bibliometric approaches to capture the existing research and assess the state of COVID-19 research across different research landscapes (e.g., health sciences, life sciences, physical sciences, social sciences, and humanities). Their findings showed that most COVID-19 research has been performed in the field of health sciences, followed by life sciences, physical sciences, and social sciences and humanities. Results from the keyword co-occurrence analysis revealed that health sciences research on COVID-19 tended to focus on health consequences, whereas the life sciences research on the subject tended to focus on drug efficiency. Moreover, physical sciences research tended to focus on environmental consequences, and social sciences and humanities research was largely oriented towards socio-economic consequences.

Similarly, Rodrigues et al. ( 2020 ) carried out a bibliometric analysis of COVID-19 related studies from a management perspective in order to elucidate how scientific research and education arrive at solutions to the pandemic crisis and the post-COVID-19 era. In line with Aristovnik et al.’s ( 2020 ) findings, Rodrigues et al. ( 2020 ) reported that most of the published research on this subject has fallen under the field of health sciences, leaving education as an under-researched area of inquiry. The content analysis they performed in their study also found a special emphasis on qualitative research. The descriptive and content analysis yielded two major strands of studies: (1) online education and (2) COVID-19 and education, business, economics, and management. The online education strand focused on the issue of technological anxiety caused by online classes, the feeling of belonging to an academic community, and feedback.

Lastly, Bond ( 2020 ) conducted a rapid review of K-12 research undertaken in the first seven months of the COVID-19 pandemic to identify successes and challenges and to offer recommendations for the future. From a search of K-12 research on the Web of Science, Scopus, EBSCOHost, the Microsoft Academic, and the COVID-19 living systematic map, 90 studies were identified and analyzed. The findings revealed that the reviewed research has focused predominantly on the challenges to shifting to ERE, teacher digital competencies and digital infrastructure, teacher ICT skills, parent engagement in learning, and students’ health and well-being. The review highlighted the need for straightforward communication between schools and families to inform families about learning activities and to promote interactivity between students. Teachers were also encouraged to develop their professional networks to increase motivation and support amongst themselves and to include opportunities for both synchronous and asynchronous interaction for promoting student engagement when using technology. Bond ( 2020 ) reported that the reviewed studies called for providing teachers with opportunities to further develop their digital technical competencies and their distance and online learning pedagogies. In a recent study that examines the impact of COVID-19 at higher education (Bozkurt, 2022 ), three broad themes from the body of research on this subject: (1) educational crisis and higher education in the new normal: resilience, adaptability, and sustainability, (2) psychological pressures, social uncertainty, and mental well-being of learners, and (3) the rise of online distance education and blended-hybrid modes. The findings of this study are similar to Mishra et al. ( 2021 ) who examined the COVID-19 pandemic from the lens of online distance education and noted that technologies for teaching and learning and psychosocial issues were emerging issues.

The aforementioned studies indicate that a great majority of research on COVID-19 has been produced in the field of health sciences, as expected. These studies nonetheless note that there is a noticeable shortage of studies dealing with the effects of the pandemic in the fields of social sciences, humanities, and education. Given the profound impact of the pandemic on learning and teaching, as well as on the related stakeholders in education, now more than ever, a greater amount of research on COVID-19 needs to be conducted in the field of education. The bibliometric studies discussed above have analyzed COVID-19 research across various fields, yielding a comparative snapshot of the research undertaken so far in different research spheres. However, despite being comprehensive, these studies did not appear to have examined a specific discipline or area of research in depth. Therefore, this bibliometric study aims to provide a focused, in-depth analysis of the COVID-19-related research in the field of education. In this regard, the main purpose of this study is to identify research patterns and trends in the field of education by examining COVID-19-related research papers. The study sought to answer the following research questions:

What are the thematic patterns in the title, abstract, and keywords of the publications on COVID-19 and education?

What are the citation trends in the references of the sampled publications on COVID-19 and education?

Methodology

This study used data mining and analytic approaches (Fayyad et al., 2002 ) to examine bibliometric patterns and trends. More specifically, social network analysis (SNA) (Hansen et al., 2020 ) was applied to examine the keywords and references, while text-mining was applied (Aggarwal & Zhai, 2012 ) to examine the titles and abstracts of the research corpus. Keywords represent the essence of an article at a micro level and for the analysis of the keywords, SNA was used. SNA “provides powerful ways to summarize networks and identify key people, [entities], or other objects that occupy strategic locations and positions within a matrix of links” (Hansen et al., 2020 , p. 6). In this regard, the keywords were analyzed based on their co-occurrences and visualized on a network graph by identifying the significant keywords which were demonstrated as nodes and their relationships were demonstrated with ties. For text-mining of the titles and abstracts, the researchers performed a lexical analysis that employs “two stages of co-occurrence information extraction—semantic and relational—using a different algorithm for each stage” (Smith & Humphreys, 2006 , p. 262). Thus, text-mining analysis enabled researchers to identify the hidden patterns and visualize them on a thematic concept map. For the analysis of the references, the researchers further used SNA based on the arguments that “citing articles and cited articles are linked to each other through invisible ties, and they collaboratively and collectively build an intellectual community that can be referred to as a living network, structure, or an ecology” (Bozkurt, 2019 , p. 498). The analysis of the references enabled the researchers to identify pivotal scholarly contributions that guided and shaped the intellectual landscape. The use of multiple approaches enables the study to present a broader view, or a meta-narrative.

Sample and Inclusion Criteria

The publications included in this research met the following inclusion criteria: (1) indexed by the Scopus database, (2) written in English, and (3) had the search queries on their title (Table 1 ). The search query reflects the focus on the impact of COVID-19 on education by including common words in the field like learn , teach , or student . Truncation was also used in the search to capture all relevant literature. Narrowing down the search allowed us to exclude publications that were not education related. Scopus was selected because it is one of the largest scholarly databases, and only publications in English were selected to facilitate identification of meaningful lexical patterns through text-mining and provide a condensed view of the research. The search yielded a total of 1150 papers (articles = 887, editorials = 66, notes = 58, conference papers = 56, letters = 40, review studies = 30, book chapters = 9, short surveys = 3, books = 1).

Data Analysis and Research Procedures

This study has two phases of analysis. In the first phase, text mining was used to analyze titles and abstracts, and SNA was applied to analyze keywords. By using two different analytical approaches, the authors were able to triangulate the research findings (Thurmond, 2001 ). In this phase, using lexical algorithms, text mining analysis enabled visualizing the textual data on a thematic concept map according to semantic relationships and co-occurrences of the words (Fig.  1 ). Text mining generated a machine-based concept map by analyzing the co-occurrences and lexical relationships of textual data. Then, based on the co-occurrences and centrality metrics, SNA enabled visualizing keywords on a network graphic called sociogram (Fig.  2 ). SNA allowed researchers to visually identify the key terms on a connected network graph where keywords are represented as nodes and their relationships are represented as edges. In the first phase of the study, by synthesizing outputs of the data mining and analytic approaches, meaningful patterns of textual data were presented as seven main research themes.

Thematic concept mapping of COVID-19 and education-related papers

Social networks analysis of the keywords in COVID-19 and education-related papers

In the second phase of the study, through the examination of the references and citation patterns (e.g., citing and being cited) of the articles in the research corpus, the citation patterns were visualized on a network graphic by clusters (See Fig.  3 ) showing also chronical relationships which enabled to identify pivotal COVID-19 studies. In the second phase of the study, two new themes were identified which were in line with the themes that emerged in the first phase of the study.

Social networks analysis of the references in COVID-19 and education-related papers 2019–2020 (Only the first authors were labeled – See Appendix Fig. 4 for SNA of references covering pre-COVID-19 period)

Strengths and Limitations

This study is one of the first attempts to use bibliometric approaches benefiting from data mining and analysis techniques to better understand COVID-19 and its consequences on published educational research. By applying such an approach, a large volume of data is able to be visualized and reported. However, besides these strengths, the study also has certain limitations. First, the study uses the Scopus database, which, though being one of the largest databases, does not include all types of publications. Therefore, the publications selected for this study offer only a partial view, as there are many significant publications in gray literature (e.g., reports, briefs, blogs). Second, the study includes only publications written in English, however, with COVID-19 being a global crisis, publications in different languages would provide a complementary view and be helpful in understanding local reflections in the field of education.

Findings and Discussion

Sna and text-mining: thematic patterns in the title, abstract, and keywords of the publications.

This section reports the findings based on a thematic concept map and network graphic that were developed through text mining (Fig.  1 —Textual data composed of 186.234 words visualized according to lexical relationships and co-occurrences) and sociograms created using SNA (Fig.  2 —The top 200 keywords with highest betweenness centrality and 1577 connections among them mapped on a network graph) to visualize the data. Accordingly, seven major themes were identified by analyzing the data through text-mining and SNA: (1) the great reset, (2) digital pedagogy, (3) shifting educational landscape and emerging educational roles, (4) emergency remote education, (5) pedagogy of care, (6) social equity, equality, and injustice, and (7) future of education.

Theme 1: The Great Reset (See path Fig.  1 : lockdown  +  emergency  +  community  +  challenges  +  during  >  pandemic and impact  >  outbreak  >  coronavirus  >  pandemic and global  >  crisis  >  pandemic  >  world; See nodes on Fig.  2 : Covid19, pandemic, Coronavirus, lockdown, crisis ). The first theme in the thematic concept map and network graphic is the Great Reset. It has been relatively a short time since the World Health Organization (WHO) declared the COVID-19 a pandemic. Although vaccination had already started, the pandemic continued to have an adverse impact on the world. Ever since the start of the pandemic, people were discussing when there would be a return to normal (Bozkurt & Sharma, 2020a , b ; Xiao, 2021 ); however, as time goes by, this hope has faded, and returning to normal appears to be far into the future (Schwab & Malleret, 2020 ). The pandemic is seen as a major milestone, in the sense that a macro reset in economic, social, geopolitical, environmental, and technological fields will produce multi-faceted changes affecting almost all aspects of life (Schwab & Malleret, 2020 ). The cover of an issue of the international edition of Time Magazine reflected this idea of a great reset and presented the COVID-19 pandemic as an opportunity to transform the way we live and work (Time, 2020 ). It has been argued that the pandemic will generate the emergence of a new era, and that we will have to adapt to the changes it produces (Bozkurt & Sharma, 2020 ). For example, the industrial sector quickly embraced remote work despite its challenges, and it is possible that most industrial companies will not return to the on-site working model even after the pandemic ends (Hern, 2020 ). We can expect a high rate of similar responses in other fields, including education, where COVID-19 has already reshaped our educational systems, the way we deliver education, and pedagogical approaches.

Theme 2: Digital pedagogy (See path on Fig.  1 : distance learning  >  research  >  teacher  >  development  >  need  >  training  +  technology  +  virtual  >  digital  >  communication  >  support  >  process  >  teaching  >  online  >  learning  >  online learning  +  course  >  faculty  >  students  >  experience ; See nodes on Fig.  2 : online learning, distance learning, computer-based learning, elearning, online education, distance education, online teaching, multimedia-based learning, technology, blended learning, online, digital transformation, ICT, online classes, flexible learning, technology-enhanced learning, digitalization ). Owing to the rapid transition to online education as a result of COVID-19, digital pedagogy and teachers’ competencies in information and communication technology (ICT) integration have gained greater prominence with the unprecedented challenges teachers have faced to adapt to remote teaching and learning. The COVID-19 pandemic has unquestionably manifested the need to prepare teachers to teach online, as most of them have been forced to assume ERE roles with inadequate preparation. Studies involving the use of SNA indicate a correspondence between adapting to a digital pedagogy and the need to equip teachers with greater competency in technology and online teaching (e.g., Blume, 2020 ; König et al., 2020 ). König et al. ( 2020 ) conducted a survey-based study investigating how early career teachers have adapted to online teaching during COVID-19 school closures. Their study found that while all the teachers maintained communication with students and their parents, introduced new learning content, and provided feedback, they lacked the ability to respond to challenges requiring ICT integration, such as those related to providing quality online teaching and to conducting assessments. Likewise, Blume ( 2020 ) noted that most teachers need to acquire digital skills to implement digitally-mediated pedagogy and communication more effectively. Both study findings point to the need for building ICT-related teaching and learning competencies in initial teacher education and teacher professional development. The findings from the SNA conducted in the present study are in line with the aforementioned findings in terms of keyword analysis and overlapping themes and nodes.

Theme 3: Shifting educational landscape and emerging educational roles (See path on Fig.  1 : future > education > role > Covid19; See nodes on Fig.  2 : higher education, education, student, curriculum, university, teachers, learning, professional development, teacher education, knowledge, readiness ). The role of technology in education and human learning has been essential during the COVID-19 pandemic. Technology has become a prerequisite for learning and teaching during the pandemic and will likely continue to be so after it. In the rapid shift to an unprecedented mode of learning and teaching, stakeholders have had to assume different roles in the educational landscape of the new normal. For example, in a comprehensive study involving the participation of over 30 K higher education students from 62 countries conducted by Aristovnik et al. ( 2020 ), it was found that students with certain socio-demographic characteristics (male, lower living standard, from Africa or Asia) were significantly less satisfied with the changes to work/life balance created by the COVID-19 pandemic, and that female students who were facing financial problems were generally more affected by COVID-19 in their emotional life and personal circumstances. Despite the challenges posed by the pandemic, there is likely to be carry over in the post-pandemic era of some of the educational changes made during the COVID-19 times. For example, traditional lecture-based teacher-centered classes may be replaced by more student-centered online collaborative classes (Zhu & Liu, 2020 ). This may require the development and proliferation of open educational platforms that allow access to high-quality educational materials (Bozkurt et al., 2020 ) and the adoption of new roles to survive in the learning ecologies informed by digital learning pedagogies. In common with the present study, the aforementioned studies (e.g., Aristovnik et al., 2020 ; König et al., 2020 ) call for more deliberate actions to improve teacher education programs by offering training on various teaching approaches, such as blended, hybrid, flexible, and online learning, to better prepare educators for emerging roles in the post-pandemic era.

Theme 4: Emergency remote education (see path Fig.  1 : higher education  >  university  >  student  >  experience  >  remote; See nodes on Fig.  2 : Covid19, pandemic, Coronavirus, higher education, education, school closure, emergency remote teaching, emergency remote learning ). Educational institutions have undergone a rapid shift to ERE in the wake of COVID-19 (Bozkurt & Sharma, 2020a ; Bozkurt et al., 2020 ; Hodges et al., 2020 ). Although ERE is viewed as similar to distance education, they are essentially different. That is, ERE is a prompt response measure to an emergency situation or unusual circumstances, such as a global pandemic or a civil war, for a temporary period of time, whereas distance education is a planned and systematic approach to instructional design and development grounded in educational theory and practice (Bozkurt & Sharma, 2020b ). Due to the urgent nature of situations requiring ERE, it may fall short in embracing the solid pedagogical learning and teaching principles represented by distance education (Hodges et al., 2020 ). The early implementations of ERE primarily involved synchronous video-conferencing sessions that sought to imitate in-person classroom instruction. It is worth noting that educators may have heavily relied on synchronous communication to overcome certain challenges, such as the lack of available materials and planned activities for asynchronous communication. Lockdowns and school closures, which turned homes into compulsory learning environments, have posed major challenges for families and students, including scheduling, device sharing, and learner engagement in a socially distanced home learning environment (Bond, 2020 ). For example, Shim and Lee ( 2020 ) conducted a qualitative study exploring university students’ ERE experiences and reported that students complained about network instability, unilateral interactions, and reduced levels of concentration. The SNA findings clearly highlight that there has been a focus on ERE due to the school closures during the COVID-19 pandemic. It is key to adopt the best practices of ERE and to utilize them regularly in distance education (Bozkurt, 2022 ). Moreover, it is important to note that unless clear distinctions are drawn between these two different forms of distance education or virtual instruction, a series of unfortunate events in education during these COVID-19 times is very likely to take place and lead to fatal errors in instructional practices and to poor student learning outcomes.

Theme 5: Pedagogy of care (See path Fig.  1 : r ole  >  education  >  Covid19  >  care ; See nodes on Fig.  2 : Stress, anxiety, student wellbeing, coping, care, crisis management, depression ). The thematic concept map and network graphic show the psychological and emotional impact of the COVID-19 pandemic on various stakeholders, revealing that they have experienced anxiety, expressed the need for care, and sought coping strategies. A study by Baloran ( 2020 ), conducted in the southern part of the Philippines to examine college students’ knowledge, attitudes, anxiety, and personal coping strategies during the COVID-19 pandemic, found that the majority of the students experienced anxiety during the lockdown and worried about food security, financial resources, social contact, and large gatherings. It was reported that the students coped with this anxiety by following protective measures, chatting with family members and friends, and motivating themselves to have a positive attitude. In a similar study, Islam et al. ( 2020 ) conducted an investigation to determine whether Bangladeshi college students experienced anxiety and depression and the factors responsible for these emotional responses. Their cross-sectional survey-based study found that a large percentage of the participants had suffered from anxiety and depression during the pandemic. Academic and professional uncertainty, as well as financial insecurity, have been documented as factors contributing to the anxiety and depression among college students. Both studies point to the need for support mechanisms to be established by higher education institutions in order to ensure student wellbeing, provide them with care, and help them to cope with stress, anxiety, and depression. Talidong and Toquero ( 2020 ) reported that, in addition to students’ well-being and care, teachers’ perceptions and experiences of stress and anxiety during the quarantine period need to be taken into account. The authors found that teachers were worried about the safety of their loved ones and were susceptible to anxiety but tended to follow the preventive policies. A pedagogy of care has been presented as an approach that would effectively allow educators to plan more supportive teaching practices during the pandemic by fostering clear and prompt communication with students and their families and taking into consideration learner needs in lesson planning (e.g., Karakaya, 2021 ; Robinson et al., 2020 ). Here it is important to stress that a pedagogy of care is a multifaceted concept, one that involves the concepts of social equity, equality, and injustice.

Theme 6: Social equity, equality, and injustice (See path on Fig.  1 : Impact  >  outbreak  >  coronavirus  >  pandemic  >  social ; See nodes on Fig.  2 : Support, equity, social justice, digital divide, inequality, social support ). One of the more significant impacts of COVID-19 has been the deepening of the existing social injustices around the world (Oldekop et al., 2020 ; Williamson et al., 2020 ). Long-term school closures have deteriorated social bonds and adversely affected health issues, poverty, economy, food insecurity, and digital divide (Van Lancker & Parolin, 2020 ). Regarding the digital divide, there has been a major disparity in access to devices and data connectivity between high-income and low-income populations increasing the digital divide, social injustice, and inequality in the world (Bozkurt et al., 2020 ). In line with the SNA findings, the digital divide, manifesting itself most visibly in the inadequacy and insufficiency of digital devices and lack of high-speed Internet, can easily result in widespread inequalities. As such, the disparities between low and high socio-economic status families and school districts in terms of digital pedagogy inequality may deepen as teachers in affluent schools are more likely to offer a wide range of online learning activities and thereby secure better student engagement, participation, and interaction (Greenhow et al., 2020 ). These findings demonstrate that social inequities have been sharpened by the unfortunate disparities imposed by the COVID-19, thus requiring us to reimagine a future that mitigates such concerns.

Theme 7: Future of education (See word path on Fig.  1 : Future  >  education  >  Covid19  >  pandemic  >  changes and pandemic  >  coronavirus, outbreak, impact  >  world ; See nodes on Fig.  2 : Sustainability, resilience, uncertainty, sdg4). Most significantly, COVID-19 the pandemic has shown the entire world that teachers and schools are invaluable resources and execute critical roles in society. Beyond that, with the compulsory changes resulting from the pandemic, it is evident that teaching and learning environments are not exclusive to brick-and-mortar classrooms. Digital technologies, being at the center of teaching and learning during the pandemic period, have been viewed as a pivotal agent in leveraging how learning takes place beyond the classroom walls (Quilter-Pinner & Ambrose, 2020 ). COVID-19 has made some concerns more visible. For example, the well-being of students, teachers, and society at large has gained more importance in these times of crisis. Furthermore, the need for educational technology and digital devices has compounded and amplified social inequities (Pelletier et al., 2021 ; West & Allen, 2020 ). Despite its global challenges, the need for technology and digital devices has highlighted some advantages that are likely to shape the future of education, particularly those related to the benefits of educational technology. For example, online learning could provide a more flexible, informal, self-paced learning environment for students (Adedoyin & Soykan, 2020 ). However, it also bears the risk of minimizing social interaction, as working in shared office environments has shifted to working alone in home-office settings. In this respect, the transformation of online education must involve a particular emphasis on sustaining interactivity through technology (Dwivedi et al., 2020 ). In view of the findings of the aforementioned studies, our text-mining and SNA findings suggest that the COVID-19 impositions may strongly shape the future of education and how learning takes place.

In summary, these themes extracted from the text-mining and SNA point to a significant milestone in the history of humanity, a multi-faceted reset that will affect many fields of life, from education and economics to sociology and lifestyle. The resulting themes have revealed that our natural response to an emerging worldwide situation shifted the educational landscape. The early response of the educational system was emergency-based and emphasized the continuance of in-person instruction via synchronous learning technologies. The subsequent response foregrounded the significance of digitally mediated learning pedagogy, related teacher competencies, and professional development. As various stakeholders (e.g., students, teachers, parents) have experienced a heightened level of anxiety and stress, an emerging strand of research has highlighted the need for care-based and trauma-informed pedagogies as a response to the side effects of the pandemic. In addition, as the global pandemic has made systemic impairments, such as social injustice and inequity, more visible, an important line of research has emerged on how social justice can be ensured given the challenges caused by the pandemic. Lastly, a sizable amount of research indicates that although the COVID-19 pandemic has imposed unprecedented challenges to our personal, educational, and social lives, it has also taught us how to respond to future crises in a timely, technologically-ready, pedagogically appropriate, and inclusive manner.

SNA: Citation Trends in the References of the Sampled Publications

The trends identified through SNA in citation patterns indicate two lines of thematic clusters (see Fig.  3 -A network graph depicting the citing and being cited patterns in the research corpus. Node sizes were defined by their citation count and betweenness centrality.). These clusters align with the results of the analysis of the titles, abstracts, and keywords of the sampled publications and forge the earlier themes (Theme 4: Emergency remote education and Theme 5: Pedagogy of care).

Thematic Cluster 1: The first cluster centers on the abilities of educational response, emergency remote education affordances, and continuity of education (Bozkurt & Sharma, 2020a ; Crawford et al., 2020 ; Hodges et al., 2020 ) to mitigate the impact of COVID-19 on education, especially for more vulnerable and disadvantaged groups (UNESCO, 2020 ; Viner et al., 2020 ). The thematic cluster one agrees with the theme four emergency remote education . The first trend line (See red line in Fig.  3 ) shows that the education system is vulnerable to external threats. Considering that interruption of education is not exclusive to pandemics – for example, political crises have also caused disruptions (Rapp et al., 2016 ) – it is clear that coping mechanisms are needed to ensure the continuity of education under all conditions. In this case, we need to reimagine and recalibrate education to make it resilient, flexible, and adaptive, not only to ensure the continuity of education, but also to ensure social justice, equity, and equality. Given that online education has its own limitations (e.g., it is restricted to online tools and infrastructures), we need to identify alternative entry points for those who do not have digital devices or lack access to the internet.

Thematic Cluster 2: The second cluster centers on the psychological impact of COVID-19 on learners, who during these times suffered a sense of uncertainty (Bozkurt, & Sharma, 2021 ; Cao et al., 2020 ; Rose, 2020 ; Sahu, 2020 ) which suggest that learners are experiencing difficult times that can result in psychological and mental problems. The thematic cluster two agrees with theme five which is pedagogy of care . Therefore, it can be argued that learners' psychological and emotional states should be a top priority. Brooks et al. ( 2020 ) reported the potential of post-traumatic issues with long-lasting effects, on top of the trauma that has already been suffered during the COVID-19 pandemic. In other words, the effects of the COVID-19 crisis may prove to extend beyond their current state and add long-term challenges. Additionally, it has further been reported that the socio-economic effects of the pandemic (Nicola et al., 2020 ) may cause inequality and inequity in educational communities (Beaunoyer et al., 2020 ). The research also shows that learners’ achievement gaps are positively associated with psychological issues, while support and care are negatively associated with their traumatic states (Cao et al., 2020 ). In this context, the second thematic cluster reveals that researchers have seriously considered the psychological and emotional needs of learners in their publications. Care (Noddings, 1984 ) and that trauma-informed pedagogy (Imad, 2020 ) can be a guideline during and after the COVID-19 pandemic. It is quite clear that learners have experienced educational loss (e.g., drop-outs, achievement gaps, academic procrastination, etc.), as well as social and emotional impairments (e.g., fear, frustration, confusion, anxiety, sense of isolation, death of loved ones, etc.). Therefore, we need to critically approach the situation, focusing first on healing our social and emotional losses, and then, on the educational losses. As Bozkurt and Sharma ( 2020a ) put it:

“What we teach in these times can have secondary importance. We have to keep in mind that students will remember not the educational content delivered, but how they felt during these hard times. With an empathetic approach, the story will not center on how to successfully deliver educational content, but it will be on how learners narrate these times” (p. iv).

Conclusion and Suggestions

The results from this study indicate that quick adaptability and flexibility have been key to surviving the substantial challenges generated by COVID-19. However, extreme demands on flexibility have taken a toll on human well-being and have exacerbated systemic issues like inequity and inequality. Using data mining that involved network analysis and text mining as analytical tools, this research provides a panoramic picture of the COVID-19-related themes educational researchers have addressed in their work. A sample of 1150 references yielded seven themes, which served to provide a comprehensive meta-narrative about COVID-19 and its impact on education.

A portion of the sampled publications focused on what we refer to as the great reset , highlighting the challenges that the emergency lockdown brought to the world. A publication pattern centered around digital pedagogy posited distance and online learning as key components and identified the need for teacher training. Given the need for adaptability, a third theme revealed the demand for professional development in higher education and a future shift in educational roles. It can be recommended that future research investigate institutional policy changes and the adaptation to these changes in renewed educational roles. The ERE theme centered on the lack of preparation in instituting the forced changes brought about by the COVID-19 pandemic. The publications related to this theme revealed that the COVID-19 pandemic uncovered silent threads in educational environments, like depression, inequality, and injustice. A pedagogy of care has been developed with the aim of reducing anxiety and providing support through coping strategies. These research patterns indicate that the future of education demands sustainability and resilience in the face of uncertainty.

Results of the thematic analysis of citation patterns (Fig.  3 ) overlapped with two of the themes found in our thematic concept map (Fig.  1 ) and network graphic (Fig.  2 ). It was shown that researchers have emphasized the continuity of education and the psychological effects of the COVID-19 crisis on learners. Creating coping strategies to deal with global crises (e.g., pandemics, political upheavals, natural disasters) has been shown to be a priority for educational researchers. The pedagogy of resilience (Purdue University Innovative learning, n.d. ) provides governments, institutions, and instructors with an alternative tool to applying to their contexts in the face of hardship. Furthermore, prioritizing the psychological long-term effects of the crisis in learners could alleviate achievement gaps. We recommend that researchers support grieving learners through care (Noddings, 1984 ) and trauma-informed pedagogy (Imad, 2020 ). Our resilience and empathy will reflect our preparedness for impending crises. The thematic analysis of citation patterns (1: educational response, emergency remote education affordances, and continuity of education; 2: psychological impact of COVID-19) further indicates suggestions for future instructional/learning designers. Freire ( 1985 ) argues that to transform the world we need to humanize it. Supporting that argument, the need for human-centered pedagogical approaches (Robinson et al., 2020 ) by considering learning a multifaceted process (Hodges et al., 2021 ) for well-designed learning experiences (Moore et al., 2021 ) is a requirement and instructional/learning designers have an important responsibility not only to design courses but an entire learning ecosystem where diversity, sensitivity, and inclusivity are prioritized.

ERE is not a representative feature in the field of online education or distance education but rather, a forced reaction to extraordinary circumstances in education. The increasing confusion between the practice of ERE and online learning could have catastrophic consequences in learners' outcomes, teachers' instructional practices, and institutional policies. Researchers, educators, and policymakers must work cooperatively and be guided by sound work in the field of distance learning to design nourishing educational environments that serve students’ best interests.

In this study, text mining and social network analysis were demonstrated to be powerful tools for exploring and visualizing patterns in COVID-19-related educational research. However, a more in-depth examination is still needed to synthesize effective strategies that can be used to support us in future crises. Systematic reviews that use classical manual coding techniques may take more time but increase our understanding of a phenomenon and help us to develop specific action plans. Future systematic reviews can use the seven themes identified in this study to analyze primary studies and find strategies that counteract the survival of the fittest mindset to ensure that no student is left behind.

Data Availability

The dataset is available from the authors upon request.

Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: The challenges and opportunities. Interactive Learning Environments . https://doi.org/10.1080/10494820.2020.1813180

Article   Google Scholar  

Aggarwal, C. C., & Zhai, C. (Eds.). (2012). Mining text data. Springer Science & Business Media. https://doi.org/10.1007/978-1-4614-3223 -4

Aristovnik, A., Keržič, D., Ravšelj, D., Tomaževič, N., & Umek, L. (2020). Impacts of the COVID-19 pandemic on life of higher education students: A global perspective. Sustainability, 12 (20), 8438. https://doi.org/10.3390/su12208438

Ayebi-Arthur, K. (2017). E-learning, resilience and change in higher education: Helping a university cope after a natural disaster. E-Learning and Digital Media, 14 (5), 259–274. https://doi.org/10.1177/2042753017751712

Baker, V. L. (2020, March 25). How colleges can better help faculty during the pandemic . Inside Higher Ed.  https://www.insidehighered.com/views/2020/03/25/recommendations-how-colleges-can-better-support-their-faculty-during-covid-19 . Accessed 15 Apr 2022

Baloran, E. T. (2020). Knowledge, attitudes, anxiety, and coping strategies of students during COVID-19 pandemic. Journal of Loss and Trauma, 25 (8), 635–642. https://doi.org/10.1080/15325024.2020.1769300

Beaunoyer, E., Dupéré, S., & Guitton, M. J. (2020). COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies. Computers in Human Behavior, 111 , 106424. https://doi.org/10.1016/j.chb.2020.106424

Blume, C. (2020). German Teachers’ Digital Habitus and Their Pandemic Pedagogy. Postdigital Science and Education, 2 (3), 879–905. https://doi.org/10.1007/s42438-020-00174-9

Bond, M. (2020). Schools and emergency remote education during the COVID-19 pandemic: A living rapid systematic review. Asian Journal of Distance Education, 15 (2), 191–247. https://doi.org/10.5281/zenodo.4425683

Bozkurt, A. (2019). Intellectual roots of distance education: A progressive knowledge domain analysis. Distance Education, 40 (4), 497–514. https://doi.org/10.1080/01587919.2019.1681894

Bozkurt, A. (2022). Resilience, adaptability, and sustainability of higher education: A systematic mapping study on the impact of the coronavirus (COVID-19) pandemic and the transition to the new normal. Journal of Learning for Development (JL4D), 9 (1), 1–16. https://doi.org/10.5281/zenodo.6370948

Bozkurt, A., & Sharma, R. C. (2020a). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15 (1), i–vi. https://doi.org/10.5281/zenodo.3778083

Bozkurt, A., & Sharma, R. C. (2020b). Education in normal, new normal, and next normal: Observations from the past, insights from the present and projections for the future. Asian Journal of Distance Education, 15 (2), i–x. https://doi.org/10.5281/zenodo.4362664

Bozkurt, A., & Sharma, R. C. (2021). On the verge of a new renaissance: Care and empathy oriented, human-centered pandemic pedagogy. Asian Journal of Distance Education, 16 (1), i–vii. https://doi.org/10.5281/zenodo.5070496

Bozkurt, A., Jung, I., Xiao, J., Vladimirschi, V., Schuwer, R., Egorov, G., Lambert, S. R., Al-Freih, M., Pete, J., Olcott, D., Jr., Rodes, V., Aranciaga, I., Bali, M., Alvarez, A. V., Jr., Roberts, J., Pazurek, A., Raffaghelli, J. E., Panagiotou, N., de Coëtlogon, P., … Paskevicius, M. (2020). A global outlook to the interruption of education due to COVID-19 pandemic: Navigating in a time of uncertainty and crisis. Asian Journal of Distance Education, 15 (1), 1–126. https://doi.org/10.5281/zenodo.3878572

Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020). The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. The Lancet, 395 (10227), 912–920. https://doi.org/10.1016/S0140-6736(20)30460-8

Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., & Zheng, J. (2020). The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Research, 287 , 112934. https://doi.org/10.1016/j.psychres.2020.112934

Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., ... & Lam, S. (2020). COVID-19: 20 countries' higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, 3 (1), 1–20. https://doi.org/10.37074/jalt.2020.3.1.7

Czerniewicz, L., Trotter, H., & Haupt, G. (2019). Online teaching in response to student protests and campus shutdowns: Academics’ perspectives. International Journal of Educational Technology in Higher Education, 16 (1), 43. https://doi.org/10.1186/s41239-019-0170-1

DePietro, A. (2020). Here’s a look at the impact of coronavirus (COVID-19) on colleges and universities in the U.S. Forbes.  https://www.forbes.com/sites/andrewdepietro/2020/04/30/impact-coronavirus-covid-19-colleges-universities/?sh=20a7121461a6 . Accessed 15 Apr 2022

Dwivedi, Y. K., Hughes, D. L., Coombs, C., Constantiou, I., Duan, Y., Edwards, J. S., Gupta, B., Lal, B., Misra, S., Prashant, P., Raman, R., Rana, N. P., Sharma, S. K., & Upadhyay, N. (2020). Impact of COVID-19 pandemic on information management research and practice: Transforming education, work and life. International Journal of Information Management, 55 , 102211. https://doi.org/10.1016/j.ijinfomgt.2020.102211

Fayyad, U., Grinstein, G. G., & Wierse, A. (Eds.). (2002). Information visualization in data mining and knowledge discovery . Morgan Kaufmann.

Google Scholar  

Freire, P. (1985). The politics of education: Culture, power and liberation . Bergin & Garvey.

Book   Google Scholar  

Greenhow, C., Lewin, C., & Staud Willet, K. B. (2020). The educational response to Covid-19 across two countries: A critical examination of initial digital pedagogy adoption. Technology, Pedagogy and Education . https://doi.org/10.1080/1475939X.2020.1866654

Guitton, M. J. (2020). Cyberpsychology research and COVID-19. Computers in Human Behavior, 111 , 106357. https://doi.org/10.1016/j.chb.2020.106357

Hansen, D. L., Shneiderman, B., Smith, M. A., & Himelboim, I. (2020). Analyzing social media networks with NodeXL: Insights from a connected world (2nd ed.). Morgan Kaufmann.

Hern, A. (2020). Covid19 could cause permanent shift towards home working. The Guardian.  http://www.miamidadetpo.org/library/2020-03-13-uk-covid19-could-cause-permanent-shift-towards-home-working.pdf . Accessed 15 Apr 2022

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning . EDUCAUSE Review.  https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning . Accessed 15 Apr 2022

Hodges, C. B., Moore, S. L., Lockee, B. B., Aaron Bond, M., Jewett, A. (2021). An Instructional Design Process for Emergency Remote Teaching. In Burgos, D., Tlili, A., Tabacco, A. (Eds), Radical Solutions for Education in a Crisis Context. Lecture Notes in Educational Technology (pp. 37–51). Singapore: Springer. https://doi.org/10.1007/978-981-15-7869-4_3

Imad, M. (2020). Leveraging the neuroscience of now. Inside Higher Ed .  https://www.insidehighered.com/advice/2020/06/03/seven-recommendations-helping-students-thrive-times-trauma . Accessed 15 Apr 2022

Islam, M. A., Barna, S. D., Raihan, H., Khan, M. N. A., & Hossain, M. T. (2020). Depression and anxiety among university students during the COVID-19 pandemic in Bangladesh: A web-based cross-sectional survey. PLoS One, 15 (8), e0238162. https://doi.org/10.1371/journal.pone.0238162

Johnson, N., Veletsianos, G., & Seaman, J. (2020). U.S. faculty and administrators’ experiences and approaches in the early weeks of the COVID-19 pandemic. Online Learning, 24 (2), 6–21. https://doi.org/10.24059/olj.v24i2.2285

Karakaya, K. (2021). Design considerations in emergency remote teaching during the COVID-19 pandemic: A human-centered approach. Education Technology Research and Development, 69 , 295–299. https://doi.org/10.1007/s11423-020-09884-0

König, J., Jäger-Biela, D. J., & Glutsch, N. (2020). Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education, 43 (4), 608–622. https://doi.org/10.1080/02619768.2020.1809650

Korkmaz, G., & Toraman, Ç. (2020). Are we ready for the post-COVID-19 educational practice? An investigation into what educators think as to online learning. International Journal of Technology in Education and Science, 4 (4), 293–309. https://doi.org/10.46328/ijtes.v4i4.110

Lorenzo, G. (2008). The Sloan Semester. Journal of Asynchronous Learning Networks, 12 (2), 5–40. https://doi.org/10.24059/olj.v12i2.1693

Mishra, S., Sahoo, S., & Pandey, S. (2021). Research trends in online distance learning during the COVID-19 pandemic. Distance Education, 42 (4), 494–519. https://doi.org/10.1080/01587919.2021.1986373

Moore, S., Trust, T., Lockee, B. B., Bond, A., & Hodges, C. (2021). One year later... and counting: Reflections on emergency remote teaching and online learning. EDUCAUSE Review.  https://er.educause.edu/articles/2021/11/one-year-later-and-counting-reflections-on-emergency-remote-teaching-and-online-learning . Accessed 15 Apr 2022

Nicola, M., Alsafi, Z., Sohrabi, C., Kerwan, A., Al-Jabir, A., Iosifidis, C., ... & Agha, R. (2020). The socio-economic implications of the coronavirus and COVID-19 pandemic: A review. International Journal of Surgery, 78 , 185-193. https://doi.org/10.1016/j.ijsu.2020.04.018

Noddings, N. (1984). Caring: A feminine approach to ethics . Moral Education.

Oldekop, J. A., Horner, R., Hulme, D., Adhikari, R., Agarwal, B., ... Zheng, Y. (2020). Covid-19 and the case for global development. World Development, 134 , 105044.

Pelletier, K., Brown, M., Brooks, D. C., McCormack, M., Reeves, J., Arbino, N., Bozkurt, A., Crawford, S., Czerniewicz, L., Gibson, R., Linder, K., Mason, J., & Mondelli, V. (2021). 2021 EDUCAUSE Horizon Report Teaching and Learning Edition . EDUCAUSE.  https://www.learntechlib.org/p/219489/ . Accessed 15 Apr 2022

Purdue University Innovative Learning. (n.d.). Hy-flex and resilient pedagogy resources.  https://www.purdue.edu/innovativelearning/teaching-remotely/pedagogy.aspx . Accessed 15 Apr 2022 

Quilter-Pinner, H., & Ambrose, A. (2020). The new normal: The future of education after Covid-19 . The Institute for Public Policy Research.

Rapp, C., Gülbahar, Y., & Adnan, M. (2016). e-Tutor: A multilingual open educational resource for faculty development to teach online. International Review of Research in Open and Distributed Learning, 17 (5), 284–289. https://doi.org/10.19173/irrodl.v17i5.2783

Robinson, H., Al-Freih, M., & Kilgore, W. (2020). Designing with care: Towards a care-centered model for online learning design. The International Journal of Information and Learning Technology, 37 (3), 99–108. https://doi.org/10.1108/IJILT-10-2019-0098

Rodrigues, M., Franco, M., & Silva, R. (2020). COVID-19 and Disruption in Management and Education Academics: Bibliometric Mapping and Analysis. Sustainability, 12 (18), 7362. https://doi.org/10.3390/su12187362

Rose, S. (2020). Medical student education in the time of COVID-19. JAMA, 323 (21), 2131–2132. https://doi.org/10.1001/jama.2020.5227

Sahu, P. (2020). Closure of universities due to coronavirus disease 2019 (COVID-19): impact on education and mental health of students and academic staff. Cureus, 12 (4). https://doi.org/10.7759/cureus.7541

Schwab, M., & Malleret, T. (2020). Covid-19: The great reset . World Economic Forum.

Shim, T. E., & Lee, S. Y. (2020). College students’ experience of emergency remote teaching due to COVID-19. Children and Youth Services Review, 119 , 105578. https://doi.org/10.1016/j.childyouth.2020.105578

Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behavior Research Methods, 38 (2), 262–279. https://doi.org/10.3758/bf03192778

Suleri, J. (2020). Learners’ experience and expectations during and post COVID-19 in higher education. Research in Hospitality Management, 10 (2), 91–96. https://doi.org/10.1080/22243534.2020.1869463

Talidong, K. J. B., & Toquero, C. M. D. (2020). Philippine teachers’ practices to deal with anxiety amid COVID-19. Journal of Loss and Trauma, 25 (6–7), 573–579. https://doi.org/10.1080/15325024.2020.1759225

Thurmond, V. A. (2001). The point of triangulation. Journal of Nursing Scholarship, 33 (3), 253–258. https://doi.org/10.1111/j.1547-5069.2001.00253.x

Time (2020). The great reset: cover image.  https://time.com/collection/great-reset/ . Accessed 15 Apr 2022

Tull, S., Dabner, N., & Ayebi-Arthur, K. (2017). Social media and e-learning in response to seismic events: Resilient practices. Journal of Open, Flexible & Distance Learning , 21 (1), 63–76.  http://www.jofdl.nz/index.php/JOFDL/article/view/405 . Accessed 15 Apr 2022

UNESCO. (2020). COVID-19 education response.  https://en.unesco.org/covid19/educationresponse/ . Accessed 15 Apr 2022

Unger, S., & Meiran, W. R. (2020). Student attitudes towards online education during the COVID-19 viral outbreak of 2020: Distance learning in a time of social distance. International Journal of Technology in Education and Science, 4 (4), 256–266. https://doi.org/10.46328/ijtes.v4i4.107

Van Lancker, W., & Parolin, Z. (2020). COVID-19, school closures, and child poverty: A social crisis in the making [published online ahead of print, 2020 Apr 7]. T he Lancet Public Health, 5 (5), e243–e244. https://doi.org/10.1016/S2468-2667(20)30084-0

Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, C., ... & Booy, R. (2020). School closure and management practices during coronavirus outbreaks including COVID-19: A rapid systematic review. The Lancet Child & Adolescent Health, 4 (5), 397-404. https://doi.org/10.1016/S2352-4642(20)30095-X

West, D., & Allen, J. (2020). How to address inequality exposed by the COVID-19 pandemic. Tech Crunch .  https://techcrunch.com/2020/10/27/how-to-address-inequality-exposed-by-the-covid-19-pandemic/ . Accessed 15 Apr 2022

Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45 (2), 107–114. https://doi.org/10.1080/17439884.2020.1761641

Xiao, J. (2021). From equality to equity to justice: Should online education be the new normal in education?. In Bozkurt, A. (Eds.), Handbook of research on emerging pedagogies for the future of education: Trauma-informed, care, and pandemic pedagogy (pp. 1–15). IGI Global. https://doi.org/10.4018/978-1-7998-7275-7.ch001

Zhu, X., & Liu, J. (2020). Education in and after Covid-19: Immediate responses and long-term visions. Postdigital Science and Education, 2 , 695–699. https://doi.org/10.1007/s42438-020-00126-3

Download references

Acknowledgements

This paper is dedicated to all educators and instructional/learning designers who ensured the continuity of education during the tough times of the COVID-19 pandemic.

This article is produced as a part of the 2020 AECT Mentoring Program.

This paper is supported by Anadolu University, Scientific Research Commission with grant no: 2106E084.

Author information

Authors and affiliations.

Distance Education Department, Anadolu University, Eskişehir, Turkey

Aras Bozkurt

Department of English Studies, University of South Africa, Pretoria, South Africa

Anadolu Üniversitesi, Açıköğretim Fakültesi, Kat:7, Oda:702, 26470, Tepebaşı, Eskişehir, Turkey

Applied Linguistics & Technology Department, Iowa State University, Ames, IA, USA

Kadir Karakaya

Educational Psychology, Learning Sciences, University of Oklahoma, Norman, OK, USA

Educational Technology & Human-Computer Interaction, Iowa State University, Ames, IA, USA

Özlem Karakaya

Curriculum and Instruction, Learning Design and Technology, Purdue University, West Lafayette, IN, USA

Daniela Castellanos-Reyes

You can also search for this author in PubMed   Google Scholar

Contributions

AB: Conceptualization, Methodology, Software, Formal analysis, Investigation, Resources, Data Curation, Writing—Original Draft, Writing—Review & Editing, Visualization, Funding acquisition.; KK: Conceptualization, Investigation, Writing—Original Draft, Writing—Review & Editing.; MT: Conceptualization, Investigation, Writing—Original Draft, Writing—Review & Editing.; ÖK: Conceptualization, Investigation, Writing—Original Draft, Writing—Review & Editing.; DCR: Conceptualization, Investigation, Writing—Original Draft, Writing—Review & Editing.

Corresponding author

Correspondence to Aras Bozkurt .

Ethics declarations

Ethics approval and consent to participate.

This is a systematic review study and exempt from ethical approval.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

SNA of references covering pre-COVID-19 period (Only the first authors were labeled)

Rights and permissions

Reprints and permissions

About this article

Bozkurt, A., Karakaya, K., Turk, M. et al. The Impact of COVID-19 on Education: A Meta-Narrative Review. TechTrends 66 , 883–896 (2022). https://doi.org/10.1007/s11528-022-00759-0

Download citation

Accepted : 22 June 2022

Published : 05 July 2022

Issue Date : September 2022

DOI : https://doi.org/10.1007/s11528-022-00759-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus pandemic
• Education during the pandemic
• Teaching and learning in the new normal
• Impact of the COVID-19 pandemic
• Find a journal
• Publish with us
• Track your research

PERSPECTIVE article

The psychological and social impact of covid-19: new perspectives of well-being.

A commentary has been posted on this article:

Commentary: The psychological and social impact of COVID-19: New perspectives of well-being

• Read general commentary

• 1 Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio of Cassino, Cassino, Italy
• 2 Independent Researcher, Milan, Italy
• 3 Department of Political and Social Studies, Sociology, University of Salerno, Fisciano, Italy

The recent Covid-19 pandemic has had significant psychological and social effects on the population. Research has highlighted the impact on psychological well-being of the most exposed groups, including children, college students, and health workers, who are more likely to develop post-traumatic stress disorder, anxiety, depression, and other symptoms of distress. The social distance and the security measures have affected the relationship among people and their perception of empathy toward others. From this perspective, telepsychology and technological devices assume important roles to decrease the negative effects of the pandemic. These tools present benefits that could improve psychological treatment of patients online, such as the possibility to meet from home or from the workplace, saving money and time and maintaining the relationship between therapists and patients. The aim of this paper is to show empirical data from recent studies on the effect of the pandemic and reflect on possible interventions based on technological tools.

Introduction

The Covid-19 pandemic led to a prolonged exposure to stress. As a consequence, researchers showed an increased interest in measuring social and community uneasiness in order to psychologically support the population. This increased attention might help in managing the current situation and other possible epidemics and pandemics. The security measures adopted in managing the pandemic had different consequences on individuals, according to the social role invested. Some segments of the population seem to be more exposed to the risk of anxious, depressive, and post-traumatic symptoms because they are more sensitive to stress.

The following article has two focuses of interest: (1) the evaluation of the psychological and social effects of the pandemic on the population, mostly children, college students, and health professionals; and (2) the identification of new perspectives of intervention based on digital devices and in line with the social security measures and mental health promotion. Telepsychology, for instance, is a valid tool, effective in taking charge of the psychological suffering caused by the pandemic and in preventing the chronicity of the disease. The prolonged stress could involve anxiety, depression, and the inability to manage traumatic and negative emotions. Furthermore, the constant fear of contagion affects daily life and leads to social isolation, modifying human relations.

COVID-19 and At-Risk Populations: Psychological and Social Impact of the Quarantine

Studies of pandemics faced over time, such as SARS, Ebola, H1N1, Equine Flu, and the current COVID-19, show that the psychological effects of contagion and quarantine is not limited on the fear of contracting the virus ( Barbisch et al., 2015 ). There are some elements related to the pandemic that affect more the population, such as separation from loved ones, loss of freedom, uncertainty about the advancement of the disease, and the feeling of helplessness ( Li and Wang, 2020 ; Cao et al., 2020 ). These aspects might lead to dramatic consequences ( Weir, 2020 ), such as the rise of suicides ( Kawohl and Nordt, 2020 ). Suicidal behaviors are often related to the feeling of anger associated with the stressful condition widely spread among people who lived/live in the most affected areas ( Miles, 2014 ; Suicide Awareness Voices of Education, 2020 ; Mamun and Griffiths, 2020 ). In light of these consequences, a carefully evaluation of the potential benefits of the quarantine is needed, taking into account the high psychological costs ( Day et al., 2006 ; Mazza et al., 2020 ).

As reported in a recent survey administered during the Covid-19 pandemic, children and young adults are particularly at risk of developing anxious symptoms ( Orgilés et al., 2020 ). The research involved a sample of 1,143 parents of Italian and Spanish children (range 3–18). In general, parents observed emotional and behavioral changes in their children during the quarantine: symptoms related to difficulty concentrating (76.6%), boredom (52%), irritability (39%), restlessness (38.8%), nervousness (38%), sense of loneliness (31.3%), uneasiness (30.4%), and worries (30.1%). From the comparison between the two groups—Spanish and Italian parents—it emerged that the Italian parents reported more symptoms in their children than the Spanish parents. Further data collected on a sample of college students at the time of the spread of the epidemic in China showed how anxiety levels in young adults are mediated by certain protective factors, such as living in urban areas, the economic stability of the family, and cohabitation with parents ( Cao et al., 2020 ). On the contrary, having infected relatives or acquaintances leads to a worsening in anxiety symptoms. Furthermore, the economic problems and the slowdown in academic activities are related with anxious symptoms ( Alvarez et al., 2020 ). In addition, an online survey conducted on the general population in China found that college students are more likely to experiencing stress, anxiety, and depression than others during the pandemic ( Li et al., 2020 ). These results suggest monitoring and promoting mental health of youths in order to reduce the negative impact of the quarantine ( CSTS, 2020 ; Fessell and Goleman, 2020 ; Li et al., 2020 ).

Health-care workers (HCWs) are another segment of population particularly affected by stress ( Garcia-Castrillo et al., 2020 ; Lai et al., 2020 ). HCWs are at risk to develop symptoms common in catastrophic situations, such as post-traumatic stress disorder, burnout syndrome, physical and emotional exhaustion, depersonalization, and dissociation ( Grassi and Magnani, 2000 ; Mache et al., 2012 ; Øyane et al., 2013 ). However, an epidemic presents different peculiarities compared to a catastrophic event, for instance, the stigmatizing attitudes in particular toward health professionals, who are in daily contact with the risk of infection ( Brooks et al., 2020 ). During SARS, up to 50% of health-care professionals suffered from acute psychological stress, exhaustion, and post-traumatic stress, caused by the fear of contagion of their family members and the prolonged social isolation ( Tam et al., 2004 ; Maunder et al., 2006 ).

As a consequence of the pandemic, the health professionals who were overworked suffered high level of psychophysical stress ( Mohindra et al., 2020 ). Health professionals also lived/live in daily life a traumatic condition called secondary traumatic stress disorder ( Zaffina et al., 2014 ), which describes the feeling of discomfort experienced in the helping relationship when treatments are not available for all patients and the professional must select who can access them and who cannot ( Roden-Foreman et al., 2017 ; Rana et al., 2020 ). Data from a survey on 1,257 HCWs who assisted patients in Covid-19 wards and in second- and third-line wards showed high percentages of depression (50%), anxiety (44.6%), insomnia (34%), and distress (71.5%) ( Lai et al., 2020 ). Also, the constant fear of contagion leads to obsessive thoughts ( Brooks et al., 2020 ), increasing the progressive closure of the person and reducing social relationships. In line with these results, Rossi et al. (2020) evaluated mental health outcomes among HCWs in Italy during the pandemic, confirming a high score of mental health issues, particularly among young women and front-line workers. Furthermore, Spoorthy et al. (2020) conducted a review on the gendered impact of Covid-19 and found that 68.7–85.5% of medical staff is composed of women, and the mean age ranged between 26 and 40 years. Also, women are more likely to be affect by anxiety, depression, and distress ( Lai et al., 2020 ; Zanardo et al., 2020 ). Liang et al. (2020) also found a relation between age and depressive symptoms associated with the pandemic. Indeed, the medical staff at younger ages (<30 years) reports higher self-rated depression scores and more concern about infecting their families than those of older age. Staff > 50 years of age reported increased stress due to patient’s death, the prolonged work hours, and the lack of personal protective equipment. Cai et al. (2020) also found that nurses felt more nervous compared to doctors.

As emerged by the recent literature, the promotion of psychological interventions on the specific population who is more likely to develop pathologies and suffering is needed. The Lancet Global Mental Health Commission’s observation ( Patel, 2018 ) reported that the use of digital technologies can provide mental health interventions in order to reduce anxiety and stress levels and increase self-efficacy ( Kang et al., 2020 ; Xiao et al., 2020 ).

Telepsychology: Training and Promotion of Psychological Well-Being

In order to reduce anxiety and depression symptoms widespread among the population, the World Health Organization (2019) and the Centers for Disease Control and Prevention (2020) proposed specific guidelines on the correct use of health protection with the aim to minimize the distress associated with health-care professions.

At the same time, as a consequence of the emerging issues, psychotherapists provided psychological support online, addressing the technological challenge ( Greenberg et al., 2020 ); Liu et al., 2020 ). In line with the technological progress, professional organizations promoted specific guidelines and policies related to customer protection, privacy, screening, evaluation, and development of self-help products ( Duan and Zhu, 2020 ; Zhou et al., 2020 ). Technological development in mental health foreshadows future trends that include “smart” mobile devices, cloud computing, virtual worlds, virtual reality, and electronic games in addition to the traditional psychotherapy tools. In this perspective, it is important to help future generations of psychologists and patients to collaborate in the potential growth areas, through education and training on the benefits and effectiveness of telepsychology ( Maheu et al., 2012 ).

Indeed, more awareness of the potentials of the online services is needed, exploring the main differences between the devices (chat, video-audio consultation, etc.) in order to use them in relation to the specific purposes identified by the professional. For example, the Italian Service of Online Psychology conducted a study based on a service of helpdesk on Facebook. This service guided people in asking for psychological help, working on their personal motivation. At the same time, another helpdesk on Skype provided some psychological sessions via webcam ( Gabri et al., 2015 ). In this line, telecounseling is a diffuse online method used by counselors and psychologists during the recent pandemic ( De Luca and Calabrò, 2020 ).

One of the future goals of public and private psychological organizations should be the promotion of specific training for psychologists and psychotherapists, with the following aims: (1) developing the basic skills in managing the effects of a pandemic and of emergency situations; and (2) sensitizing patients to online therapeutic relationship, providing the main rules and benefits of the process ( Stoll et al., 2020 ; Joint Task Force for the Development of Telepsychology Guidelines for Psychologists, 2013 ). On this line, a significant example is the Virginia Commonwealth University (VCU) which proposed PhDs in telepsychology, with the aim of training future psychologists in managing the psychological effects of the pandemic through an online psychology service ( Baylor et al., 2019 ). The service provided by the VCU had been effective in reducing anxiety, depression ( Sadock et al., 2017 ), and hospital recoveries ( Lanoye et al., 2017 ). As shown, telepsychology assumes a key role in the improvement of health care. Online psychological services avoid geographical barriers and are suitable to become a useful integrated tool in addition to traditional psychotherapy ( APS, 2020 ; Perrin et al., 2020 ).

Advantages of Psychological Support and Online Psychotherapy

Online psychological services provide several advantages, especially in the current situation of pandemic. First of all, online services help people in a short period of time, reducing the risk of contagion and the strong feeling of anxiety in both psychotherapists and patients, who feel uncomfortable in doing traditional psychotherapy due to the pandemic ( Békés and Aafjes-van Doorn, 2020 ). Furthermore, Pietrabissa et al. (2015) identified some of the main advantages of telepsychology, such as the decrease in waiting for the consultation, because it takes place from home or from the workplace, saving time and expense, less travel and rental costs for the office, for those who provide the service and for those who use it. As reported by the authors, online psychological services facilitate access to people who struggle to find support close to their social environment, avoiding difficulties related to mobility. Also, online services help people who have less confidence in psychotherapy. Indeed, mostly online psychotherapy takes place in one’s comfort zone, facilitating the expression of problems and feelings.

According to the situations, online services could provide a different medium. For instance, the chat is a useful tool to establish a first assessment of a person who feels uncomfortable in using video. Indeed, the online psychotherapy is perceived as more “acceptable.” Suler (2004) defined the term online disinhibition effect demonstrating how the web, unlike the real life, leads to the failure of the hierarchical relationship based on dominant-dominated among individuals; this aspect, according to the author, allows a greater sense of freedom in expressing oneself and less concern related to judgment ( ibid .). Other researchers ( Mantovani, 1995 ; Tosoni, 2004 ) have integrated to the construct of online disinhibition effect the concept of social space, emphasizing the role of the “situation,” of the “social norms” ( Brivio et al., 2010 , p. 811), of the tools (“artifacts”), and of the cyberplace, which allow different levels of interaction. Each person has a different experience of the network and several levels of disinhibition. For instance, a mild disinhibition could be a person who chooses to ask for help talking with a psychologist about their problems; while a high disinhibition could be represented by flaming, an expression of online bullying or cyberstalking.

Online psychological services should be integrated with the various territorial services in order to provide the patients local references in relation to the specific health and economic needs. Finally, the possibility for the therapist and for the patient to record the sessions via chat and in audio/video mode—with the informed consent of the participants ( Wells et al., 2015 )—provides another useful tool to compare the sessions and to underline the positive outcomes and the effectiveness of the therapeutic process. According to this perspective, online psychological support and psychotherapy become a resource for psychotherapists and patients in a co-build relationship ( Algeri et al., 2019 ).

Psychological and Social Suffering and the Empathic Process

In analyzing the psychological impact of the quarantine, the importance for individuals to feel integral part of the society emerged, an aspect often undervalued in psychological well-being. Experts of public health believe that social distancing is the better solution to prevent the spread of the virus. However, although it is not possible to predict the duration of the pandemic, we know very well the serious impact of these measures on the society, on relationships and interactions, in particular on the empathic process. In the early 90s, empathy was described as a form of identification in the psychological and physiological states of others. This definition led to a debate between the disciplines of philosophy of psychology and philosophy of the mind ( Franks, 2010 ). Willard Van Orman Quine (1908–2000) renewed attention to the debate on empathy with a thesis on the development of language and mind in the analytical philosophy. According to Quine, the attribution of the so-called intentional states, through which the psychology commonly explains human behavior, is based on empathy ( Treccani, 2020 ) and leads people to attribute beliefs, desires, and perceptions ( Quine, 1990 , 1992 , Pursuit of Truth: Revised Edition, 1992). Analyzing this aspect within the recent situation of the pandemic, an increment of antithetical positions and attitudes could be noticed. On the one hand, people identify themselves with those who suffer (neighbors, friends, relatives who are living stressful events), promoting activities such as the so-called “suspended expenses.” For instance, solidarity and humanitarian activities, food, and medicine delivery for people who are unable to go to the supermarket. On the other hand, there is a part of the population who experiences a feeling of “forced empathy.” This aspect could be also emphasized by the use of technological devices that might lead to a depersonalization of relationships, forcing the sense of closeness, at least virtually. The hyperconnection of feelings becomes a way to reduce the self-isolation and its consequences, representing the contrary of the idea of Durkheim (1858–1917), who considered society as a specific entity, built on social facts ( Durkheim, 1922 ). The sensation “to be forced to feel” could lead people to distance themselves from others after the emergency situation, incrementing social phobias.

Also, human communication is changing. The formal question “how are you?” at the beginning of a conversation is no longer just a formality, as before the pandemic. For example, the relationship between employee and the manager is different, leading to more responsibilities in listening and understanding feelings expressed during the video call, generating a forced reciprocity. Hence, the aforementioned “forced empathy” may be common in this period because the social distance and the emergency situation make people want to be heard and appreciated, and the simple question “how are you?” becomes an anchor to express fears and emotions ( Pasetti, 2020 ).

The Covid-19 pandemic has affected the way people live interpersonal relationships. The lockdown was characterized of a different organization of daily life, with an incrementation of time at home and a reduction of distance through digital devices. This period was also seen as an evolution in the concept of empathy, producing new perspectives in the study of the phenomenon according to a sociological and neurological points of view. Indeed, empathy—defined as the ability to understand and share the feelings of another—involves several elements, such as: (a) social context and historical period of the individual, (b) neurological mechanisms, and (c) psychological and behavioral responses to feelings of others. The neuro-sociological perspective analyzes the mechanisms involved in the empathic process, focusing on human communication and interpersonal relationships ( Singer and Lamm, 2009 ; Decety and Ickes, 2009 ). Specifically, in this historical period characterized by an increment in the man–machine relationship, neurosociology could become one of the principal sciences for the study of human relations and technology. “We live increasingly in a human–machine world. Anyone who doesn’t understand this, and who is not struggling to adapt to the new environment—whether they like that environment or not—is already being left behind. Adapting to the new, fast-changing, technologically enhanced context is one of the major challenges of our times. And that certainly goes for education” ( Prensky, 2012 , p. 64).

According to the abovementioned considerations, our suggestion consists in:

Primary prevention. Studying the impact of the pandemic toward an at-risk population to reduce symptoms related to stress and providing specific online psychological counseling based on the target (students, medical staff, parents, and teachers).

Secondary prevention. Overcoming the limitations of the human interaction based on digital devices: (1) developing new spaces of inter- and intrasocial communication and new tools of support and psychological treatment, reproducing the multisensory experienced during the face-to-face interaction (Virtual Reality, holograms, serious game etc.); (2) training the next generation of psychotherapists in managing online devices and in implementing their adaptive and personal skills; and (3) sensitizing the general population on telepsychology and its advantages.

Research according to the neurosociological perspective . Studying human interaction mediated by new technologies and the role of empathy, associating neuroscience, sociology, and psychology.

Author Contributions

VS, DA, and VA conceptualized the contribution. VS wrote the paper, reviewed the manuscript, and provided the critical revision processes as PI. All authors approved the submission of the manuscript.

This work did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Algeri, D., Gabri, S., and Mazzucchelli, L. (2019). Consulenza psicologica online. Esperienze pratiche, linee guida e ambiti di intervento. Firenze: Giunti Editore.

Google Scholar

Alvarez, F., Argente, D., and Lippi, F. (2020). A simple planning problem for Covid-19 lockdown. Covid Econ. 14, 1–33. doi: 10.3386/w26981

CrossRef Full Text | Google Scholar

APS (2020). Psychologists welcome health fund telehealth support of Australians’ mental health during COVID-19 outbreak. Australia: Tratto da Australian Psuchology Society.

Barbisch, D., Koenig, K., and Shih, F. (2015). Is there a case for quarantine? Perspectives from SARS to Ebola . Dis. Med. Pub. Health Prepar. 9, 547–553. doi: 10.1017/dmp.2015.38

PubMed Abstract | CrossRef Full Text | Google Scholar

Baylor, C., Burns, M., McDonough, K., Mach, H., and Yorkstona, K. (2019). Teaching Medical Students Skills for Effective Communication With Patients Who Have Communication Disorders. Am. J. Spe. Lang. Pathol. 28, 155–164. doi: 10.1044/2018_ajslp-18-0130

Békés, V., and Aafjes-van Doorn, K. (2020). Psychotherapists’ attitudes toward online therapy during the COVID-19 pandemic. J. Psychother. Integr. 30, 238–247. doi: 10.1037/int0000214

Brivio, E., Ibarra, F., Galimberti, C., and Cilento. (2010). “An Integrated Approach to Interactions in Cyberplaces: The Presentation of Self in Blogs,” in Handbook of Research on Discourse Behavior and Digital Communication: Language Structures and Social Interaction , eds E. Brivio, F. Ibarra, and C. Galimberti (Pennsylvania: Information Science Reference/IGI Global), 810–829. doi: 10.4018/978-1-61520-773-2.ch052

Brooks, S., Webster, R. S., Woodland, L., Wessely, S., Greenberg, N., et al. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 395:10227. doi: 10.1016/S0140-6736(20)30460-8

Cai, H., Tu, B., Ma, J., Chen, L., Fu, L., Jiang, Y., et al. (2020). Psychological impact and coping strategies of frontline medical staff in Hunan between January and March 2020 during the outbreak of coronavirus disease 2019 (COVID19) in Hubei. China. Med. Sci. Monit. 2020:26. doi: 10.12659/MSM.924171

Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., et al. (2020). The psychological impact of the COVID-19 epidemic on college students in China. Psych. Res. 287:112934. doi: 10.1016/j.psychres.2020.112934

Centers for Disease Control and Prevention (2020). Preparedness Tools for Healthcare Professionals and Facilities Responding to Coronavirus (COVID-19) . Available online at: https://www.cdc.gov/coronavirus/2019-ncov/hcp/preparedness-checklists.html

CSTS (2020). Immediate Psychological Responses and Associated Factors during the Initial Stage of the 2019 Coronavirus Disease (COVID-19) Epidemic among the General Population in China. Int. J. Environ. Res. Pub Health 17:1729. doi: 10.3390/ijerph17051729

Day, T., Park, A., Madras, N., Gumel, A., and Wu, J. (2006). When Is Quarantine a Useful Control Strategy for Emerging Infectious Diseases? Am. J. Epidemiol. 163, 479–485. doi: 10.1093/aje/kwj056

De Luca, R., and Calabrò, R. S. (2020). How the COVID-19 Pandemic is Changing Mental Health Disease Management: The Growing Need of Telecounseling in Italy. Innov. Clin. Neurosci. 17, 16–17.

Decety, J., and Ickes, W. (eds) (2009). The Social Neuroscience of Empathy. Cambridge, MA: MIT Press.

Duan, L., and Zhu, G. (2020). Psychological interventions for people affected by the COVID-19 epidemic. Lancet. Psych. 7, 300–302. doi: 10.1016/s2215-0366(20)30073-0

Durkheim, E. (1922). Education et Sociologie. Milano: Ledizioni.

Fessell, D., and Goleman, D. (2020). How Healthcare Personnel Can Take Care of Themselves. US: HBR.

Franks, D. (2010). Neurosociology the nexus between neuroscience and social psychology. Londra. Springer.

Gabri, S., Mazzucchelli, S., and Algeri, D. (2015). The request for psychological help in the digital age: offering counseling through chat and video counseling. E J. Psychother. 2015, 2–10.

Garcia-Castrillo, L., Petrino, R., and Leach, R. (2020). European Society For Emergency Medicine position paper on emergency medical systems’ response to COVID-19. Eur. J. Emerg. Med. 27, 174–1777. doi: 10.1097/mej.0000000000000701

Grassi, L., and Magnani, K. (2000). Psychiatric Morbidity and Burnout in the Medical Profession: An Italian Study of General Practitioners and Hospital Physicians. Psychother. Psychosom. 69, 329–334. doi: 10.1159/000012416

Greenberg, N., Docherty, M., Gnanapragasam, S., and Wessely, S. (2020). Managing mental health challenges faced by healthcare workers during covid-19 pandemic. BMJ 368:m1211. doi: 10.1136/bmj.m1211

Joint Task Force for the Development of Telepsychology Guidelines for Psychologists (2013). Guidelines for the practice of telepsychology. Am. Psychol. 68, 791–800. doi: 10.1037/a0035001

Kang, L., Ma, S., and Chen, M. (2020). Impact on mental health and perceptions of psychological care among medical and nursing staff in Wuhan during the 2019 novel coronavirus disease outbreak: A cross-sectional study. Brain Behav. Immun. 87, 11–17. doi: 10.1016/j.bbi.2020.03.028

Kawohl, W., and Nordt, C. (2020). COVID-19, unemployment, and suicide. Lancet Psych. 7, 389–390. doi: 10.1016/s2215-0366(20)30141-3

Lai, J., Ma, S., Wang, Y., Cai, Z., Hu, J., Wei, N., et al. (2020). Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Network Open 3:e203976. doi: 10.1001/jamanetworkopen.2020.3976

Lanoye, A., Stewart, K., Rybarczyk, B., Auerbach, S., Sadock, E., Aggarwal, A., et al. (2017). The impact of integrated psychological services in a safety net primary care clinic on medical utilization. J. Clin. Psychol. 73, 681–692. doi: 10.1002/jclp.22367

Li, L. Z., and Wang, S. (2020). Prevalence and predictors of general psychiatric disorders and loneliness during COVID-19 in the United Kingdom. Psych. Res. 291, 0165–1781. doi: 10.1016/j.psychres.2020.113267

Li, S., Wang, Y., Yang, Y., Lei, X., and Yang, Y. (2020). Analysis of influencing factors of anxiety and emotional disorders in children and adolescents during home isolation during the epidemic of novel coronavirus pneumonia. Chin. J. Child Heal 2020, 1–9.

Liang, Y., Chen, M., Zheng, X., and Liu, J. (2020). Screening for Chinese medical staff mental health by SDS and SAS during the outbreak of COVID-19. J. Psychosom. Res. 133, 1101–1102. doi: 10.1016/j.jpsychores.2020.110102

Liu, S., Yang, L., Zhang, C., Xiang, Y. T., Liu, Z., Hu, S., et al. (2020). Online mental health services in China during the COVID-19 outbreak. Lancet. Psych. 7, E17–E18. doi: 10.1016/S2215-0366(20)30077-8

Mache, S., Vitzthum, K., and Klapp, B. (2012). Stress, health and satisfaction of Australian and German doctors-a comparative study. World Hosp Health 48, 21–27.

Maheu, M. P., McMenamin, J., and Posen, L. (2012). Future of telepsychology, telehealth, and various technologies in psychological research and practice. Profess. Psychol. Res. Prac. 43, 613–621. doi: 10.1037/a0029458

Mamun, M. A., and Griffiths, M. D. (2020). First COVID-19 suicide case in Bangladesh due to fear of COVID-19 and xenophobia: Possible suicide prevention strategies. Asian J. Psych. 51:102073. doi: 10.1016/j.ajp.2020.102073

Mantovani, G. (1995). Comunicazione e Identità: dalle situazioni quotidiane agli ambienti virtuali. Bologna: il Mulino.

Maunder, R. G., Lancee, W. J., Balderson, K. E., Bennett, J. P., and Borgundvaag, B. (2006). Long-term psychological and occupational effects of providing hospital healthcare during SARS outbreak. Emerg. Infect. Dis. 12, 1924–1932. doi: 10.3201/eid1212.060584

Mazza, C., Ricci, E., Biondi, S., Colasanti, M., Ferracuti, S., Napoli, C., et al. (2020). Nationwide Survey of Psychological Distress among Italian People during the COVID-19 Pandemic: Immediate Psychological Responses and Associated Factors. Int. J. Environ. Res. Publ. Health 17:3165. doi: 10.3390/ijerph17093165

Miles, S. (2014). Kaci Hickox: Public Health and the Politics of Fear. Tratto da Bioethics . Available online at: http://www.bioethics.net/2014/11/kaci-hickox-public-health-and-the-politics-of-fear/ (accessed June 2, 2020).

Mohindra, R. R. R., Suri, V., Bhalla, A., and Singh, S. M. (2020). Issues relevant to mental health promotion in frontline health care providers managing quarantined/isolated COVID19 patients. Asian J. Psych. 51:102084. doi: 10.1016/j.ajp.2020.102084

Orgilés, M., Morales, A., Delvecchio, E., Mazzeschi, C., and Espada, J. (2020). Immediate psychological effects of the COVID-19 quarantine in youth from Italy and Spain. PsyArXiv 2020, 1–13. doi: 10.1017/s0033291720001841

Øyane, N. P. S., Elisabeth, M., Torbjörn, A., and Bjørn, B. (2013). Associations between night work and anxiety, depression, insomnia, sleepiness and fatigue in a sample of Norwegian nurses. PLoS One 2013:e70228. doi: 10.1371/journal.pone.0070228

Pasetti, J. (2020). Smart-working, costretti all’empatia da convenevoli forzati. Tratto da Sole24Ore . Available online at: https://alleyoop.ilsole24ore.com/2020/03/20/covid-19-empatia/?refresh_ce=1 (accessed June 3, 2020).

Patel, V. (2018). The Lancet Commission on global mental health and sustainable development. Lancet 392, 1553–1598. doi: 10.1016/S0140-6736(18)31612-X

Perrin, P., Rybarczyk, B., Pierce, B., Jones, H., Shaffer, C., and Islam, L. (2020). Rapid telepsychology deployment during the COVID-19 pandemic: A special issue commentary and lessons from primary care psychology training. Clin. Psychol. 76, 1173–1185. doi: 10.1002/jclp.22969

Pietrabissa, G., Manzoni, A., Algeri, D., Mazzucchelli, L., Carella, A., Pagnini, F., et al. (2015). Facebook Use as Access Facilitator for Consulting Psychology. Austr. Psychol. 50, 299–303. doi: 10.1111/ap.12139

Prensky, M. (2012). What ISN’T Technology Good At? Empathy for One Thing!. Educat. Technol. 52:64.

Quine, W. (1990). Pursuit of Truth. New York: Harvard University Press.

Quine, W. (1992). Pursuit of Truth: Revised Edition. New York: Harvard University Press.

Rana, W., Mukhtar, S., and Mukhtar, S. (2020). Mental health of medical workers in Pakistan during the pandemic COVID-19 outbreak. Asian J. Psych. 51:102080. doi: 10.1016/j.ajp.2020.102080

Roden-Foreman, K., Solis, J., Jones, A., Bennett, M., Roden-Foreman, J., Rainey, E., et al. (2017). Prospective Evaluation of Posttraumatic Stress Disorder and Depression in Orthopaedic Injury Patients With and Without Concomitant Traumatic Brain Injury. J. Orthop. Trauma 31, e275–e280. doi: 10.1097/BOT.0000000000000884

Rossi, R., Socci, V., and Pacitti, F. (2020). Mental Health Outcomes Among Frontline and Second-Line Health Care Workers During the Coronavirus Disease 2019 (COVID-19) Pandemic in Italy. JAMA Netw Open. 3:e2010185. doi: 10.1001/jamanetworkopen.2020.10185

Sadock, E., Perrin, P., Grinnell, R., Rybarczyk, B., and Auerbach, S. (2017). Initial and follow-up evaluations of integrated psychological services for anxiety and depression in a safety net primary care clinic. Am. Psycol. Assoc. 73, 1462–1481. doi: 10.1002/jclp.22459

Singer, T., and Lamm, C. (2009). The Social Neuroscience of Empathy. Ann. N. Y. Acad. Sci. 2009, 81–96. doi: 10.1111/j.1749-6632.2009.04418.x

Spoorthy, M. S., Pratapa, S. K., and Mahant, S. (2020). Mental health problems faced by healthcare workers due to the COVID-19 pandemic–A review. Asian J. Psych. 51:1876. doi: 10.1016/j.ajp.2020.102119

Stoll, J., Müller, J., and Trachsel, M. (2020). Ethical Issues in Online Psychotherapy: A Narrative Review. Front. Psych. 10:993. doi: 10.3389/fpsyt.2019.00993

Suicide Awareness Voices of Education (2020). Preventing Suicide During and After the COVID-19 Pandemic . Available online at: https://save.org/blog/preventing-suicide-covid-19-pandemic (accessed June 5, 2020).

Suler, J. (2004). The Online Disinhibition Effect. Cyb. Psychol. Behav. 7, 321–326. doi: 10.1089/1094931041291295

Tam, C., Pang, E., Lam, L., and Chiu, H. (2004). Severe acute respiratory syndrome (SARS) in Hong Kong in 2003: stress and psychological impact among frontline healthcare workers. Psychol. Med. 34, 1197–1204. doi: 10.1017/s0033291704002247

Tosoni, S. (2004). Identità virtuali: comunicazione mediata da computer e processi di costruzione dell’identità personale. Milano: FrancoAngeli.

Treccani (2020). Einfuhlung. Tratto da Treccani . Available online at: http://www.treccani.it/enciclopedia/einfuhlung/ (accessed June 10, 2020).

Weir, K. (2020). Grief and COVID-19: Mourning our bygonelives. Washington: American Psychological Association.

Wells, S. W., Moreno, L., Butler, E., and Glassman, L. (2015). “The informed consent process for therapeutic communication in clinical videoconferencing,” in Clinical videoconferencing in telehealth: Program development and practice , eds P. W. Tuerk and P. Shore (Berlin: Springer Inernational Publishing), 133–166. doi: 10.1007/978-3-319-08765-8_7

World Health Organization (2019). Emergency Global Supply Chain System (COVID-19) Catalogue . Available online at: https://www.who.int/publications/i/item/emergency-global-supply-chain-system-(covid-19)-catalogue (accessed June 10, 2020).

Xiao, H., Zhang, Y., Kong, D., Li, S., and Yang, N. (2020). The effects of social support on sleep quality of medical staff treating patients with coronavirus disease 2019 (COVID-19) in January and February 2020 in China. Med. Sci. Monit. 26:e923549. doi: 10.12659/MSM.923549

Zaffina, S., Camisa, V., Monducci, E., Vinci, M., Vicari, S., and Bergamaschi, A. (2014). Disturbo post traumatico da stress in operatori sanitari coinvolti in un incidente rilevante avvenuto in ambito ospedaliero. La Med. Del Lav. 105:2014.

Zanardo, V., Manghina, V., Giliberti, L., Vettore, M., Severino, L., and Straface, G. (2020). Psychological impact of COVID-19 quarantine measures in northeastern Italy on mothers in the immediate postpartum period. Gynechol. Obst. 150, 184–188. doi: 10.1002/ijgo.13249

Zhou, X., Snoswell, C. L., and Harding, L. E. (2020). The Role of Telehealth in Reducing the Mental Health Burden from COVID-19. Telemed. E Health. 26, 377–379. doi: 10.1089/tmj.2020.0068

Keywords : COVID-19, empathy, psychological disease, psychotherapy, social distancing, telepsychology

Citation: Saladino V, Algeri D and Auriemma V (2020) The Psychological and Social Impact of Covid-19: New Perspectives of Well-Being. Front. Psychol. 11:577684. doi: 10.3389/fpsyg.2020.577684

Received: 29 June 2020; Accepted: 03 September 2020; Published: 02 October 2020.

Reviewed by:

Copyright © 2020 Saladino, Algeri and Auriemma. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Valeria Saladino, [email protected] ; [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Research article
• Open access
• Published: 04 June 2021

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

• Israel Júnior Borges do Nascimento 1 , 2 ,
• Dónal P. O’Mathúna 3 , 4 ,
• Thilo Caspar von Groote 5 ,
• Hebatullah Mohamed Abdulazeem 6 ,
• Ishanka Weerasekara 7 , 8 ,
• Ana Marusic 9 ,
• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 10 ,
• Vinicius Tassoni Civile 11 ,
• Irena Zakarija-Grkovic 9 ,
• Tina Poklepovic Pericic 9 ,
• Alvaro Nagib Atallah 11 ,
• Santino Filoso 12 ,
• Nicola Luigi Bragazzi 13 &
• Milena Soriano Marcolino 1

On behalf of the International Network of Coronavirus Disease 2019 (InterNetCOVID-19)

BMC Infectious Diseases volume  21 , Article number:  525 ( 2021 ) Cite this article

18k Accesses

35 Citations

14 Altmetric

Metrics details

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as “critically low”. Identified symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

Conclusions

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Peer Review reports

The spread of the “Severe Acute Respiratory Coronavirus 2” (SARS-CoV-2), the causal agent of COVID-19, was characterized as a pandemic by the World Health Organization (WHO) in March 2020 and has triggered an international public health emergency [ 1 ]. The numbers of confirmed cases and deaths due to COVID-19 are rapidly escalating, counting in millions [ 2 ], causing massive economic strain, and escalating healthcare and public health expenses [ 3 , 4 ].

The research community has responded by publishing an impressive number of scientific reports related to COVID-19. The world was alerted to the new disease at the beginning of 2020 [ 1 ], and by mid-March 2020, more than 2000 articles had been published on COVID-19 in scholarly journals, with 25% of them containing original data [ 5 ]. The living map of COVID-19 evidence, curated by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), contained more than 40,000 records by February 2021 [ 6 ]. More than 100,000 records on PubMed were labeled as “SARS-CoV-2 literature, sequence, and clinical content” by February 2021 [ 7 ].

Due to publication speed, the research community has voiced concerns regarding the quality and reproducibility of evidence produced during the COVID-19 pandemic, warning of the potential damaging approach of “publish first, retract later” [ 8 ]. It appears that these concerns are not unfounded, as it has been reported that COVID-19 articles were overrepresented in the pool of retracted articles in 2020 [ 9 ]. These concerns about inadequate evidence are of major importance because they can lead to poor clinical practice and inappropriate policies [ 10 ].

Systematic reviews are a cornerstone of today’s evidence-informed decision-making. By synthesizing all relevant evidence regarding a particular topic, systematic reviews reflect the current scientific knowledge. Systematic reviews are considered to be at the highest level in the hierarchy of evidence and should be used to make informed decisions. However, with high numbers of systematic reviews of different scope and methodological quality being published, overviews of multiple systematic reviews that assess their methodological quality are essential [ 11 , 12 , 13 ]. An overview of systematic reviews helps identify and organize the literature and highlights areas of priority in decision-making.

In this overview of systematic reviews, we aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Methodology

Research question.

This overview’s primary objective was to summarize and critically appraise systematic reviews that assessed any type of primary clinical data from patients infected with SARS-CoV-2. Our research question was purposefully broad because we wanted to analyze as many systematic reviews as possible that were available early following the COVID-19 outbreak.

Study design

We conducted an overview of systematic reviews. The idea for this overview originated in a protocol for a systematic review submitted to PROSPERO (CRD42020170623), which indicated a plan to conduct an overview.

Overviews of systematic reviews use explicit and systematic methods for searching and identifying multiple systematic reviews addressing related research questions in the same field to extract and analyze evidence across important outcomes. Overviews of systematic reviews are in principle similar to systematic reviews of interventions, but the unit of analysis is a systematic review [ 14 , 15 , 16 ].

We used the overview methodology instead of other evidence synthesis methods to allow us to collate and appraise multiple systematic reviews on this topic, and to extract and analyze their results across relevant topics [ 17 ]. The overview and meta-analysis of systematic reviews allowed us to investigate the methodological quality of included studies, summarize results, and identify specific areas of available or limited evidence, thereby strengthening the current understanding of this novel disease and guiding future research [ 13 ].

A reporting guideline for overviews of reviews is currently under development, i.e., Preferred Reporting Items for Overviews of Reviews (PRIOR) [ 18 ]. As the PRIOR checklist is still not published, this study was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 statement [ 19 ]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [ 14 ].

Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.

Eligibility criteria

Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19 including, but not limited to, those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings, or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.

No restrictions were imposed on the design of the primary studies included within the systematic reviews, the last search date, whether the review included meta-analyses or language. Reviews related to SARS-CoV-2 and other coronaviruses were eligible, but from those reviews, we analyzed only data related to SARS-CoV-2.

No consensus definition exists for a systematic review [ 20 ], and debates continue about the defining characteristics of a systematic review [ 21 ]. Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion [ 14 ]. That is supported by a recent scoping review about guidance for overviews of systematic reviews [ 22 ].

Thus, for this study, we defined a systematic review as a research report which searched for primary research studies on a specific topic using an explicit search strategy, had a detailed description of the methods with explicit inclusion criteria provided, and provided a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Cochrane and non-Cochrane systematic reviews were considered eligible for inclusion, with or without meta-analysis, and regardless of the study design, language restriction and methodology of the included primary studies. To be eligible for inclusion, reviews had to be clearly analyzing data related to SARS-CoV-2 (associated or not with other viruses). We excluded narrative reviews without those characteristics as these are less likely to be replicable and are more prone to bias.

Scoping reviews and rapid reviews were eligible for inclusion in this overview if they met our pre-defined inclusion criteria noted above. We included reviews that addressed SARS-CoV-2 and other coronaviruses if they reported separate data regarding SARS-CoV-2.

Information sources

Nine databases were searched for eligible records published between December 1, 2019, and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.

The comprehensive search strategy for each database is provided in Additional file 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported into the Covidence platform [ 23 ]. In addition to database searches, we screened reference lists of reviews included after screening records retrieved via databases.

Study selection

All searches, screening of titles and abstracts, and record selection, were performed independently by two investigators using the Covidence platform [ 23 ]. Articles deemed potentially eligible were retrieved for full-text screening carried out independently by two investigators. Discrepancies at all stages were resolved by consensus. During the screening, records published in languages other than English were translated by a native/fluent speaker.

Data collection process

We custom designed a data extraction table for this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.

We extracted the following data: article identification data (authors’ name and journal of publication), search period, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA), we extracted journal rank (quartile) and Journal Impact Factor (JIF).

We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no), and length of stay in the intensive care unit.

The following outcomes were categorized as exploratory: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).

We also collected data on reporting guidelines and requirements for the publication of systematic reviews and meta-analyses from journal websites where included reviews were published.

Quality assessment in individual reviews

Two researchers independently assessed the reviews’ quality using the “A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2)”. We acknowledge that the AMSTAR 2 was created as “a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both” [ 24 ]. However, since AMSTAR 2 was designed for systematic reviews of intervention trials, and we included additional types of systematic reviews, we adjusted some AMSTAR 2 ratings and reported these in Additional file 2 .

Adherence to each item was rated as follows: yes, partial yes, no, or not applicable (such as when a meta-analysis was not conducted). The overall confidence in the results of the review is rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains, which are items 2, 4, 7, 9, 11, 13, 15 as defined by AMSTAR 2 authors [ 24 ]. We reported our adherence ratings for transparency of our decision with accompanying explanations, for each item, in each included review.

One of the included systematic reviews was conducted by some members of this author team [ 25 ]. This review was initially assessed independently by two authors who were not co-authors of that review to prevent the risk of bias in assessing this study.

Synthesis of results

For data synthesis, we prepared a table summarizing each systematic review. Graphs illustrating the mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths, and limitations.

For analysis of the prevalence of clinical outcomes, we extracted data on the number of events and the total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9–6), using the “metaprop” function for reviews that did not perform a meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform a meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ 2 . We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. We created forest plots using the RStudio© software, with the “metafor” package (version 2.1–0) and “forest” function.

Managing overlapping systematic reviews

Some of the included systematic reviews that address the same or similar research questions may include the same primary studies in overviews. Including such overlapping reviews may introduce bias when outcome data from the same primary study are included in the analyses of an overview multiple times. Thus, in summaries of evidence, multiple-counting of the same outcome data will give data from some primary studies too much influence [ 14 ]. In this overview, we did not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic [ 14 ]. To avoid any bias in summary estimates associated with overlapping reviews, we generated forest plots showing data from individual systematic reviews, but the results were not pooled because some primary studies were included in multiple reviews.

Our search retrieved 1063 publications, of which 175 were duplicates. Most publications were excluded after the title and abstract analysis ( n = 860). Among the 28 studies selected for full-text screening, 10 were excluded for the reasons described in Additional file 3 , and 18 were included in the final analysis (Fig. 1 ) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Reference list screening did not retrieve any additional systematic reviews.

PRISMA flow diagram

Characteristics of included reviews

Summary features of 18 systematic reviews are presented in Table 1 . They were published in 14 different journals. Only four of these journals had specific requirements for systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology, and Clinical Research in Cardiology . Two journals reported that they published only invited reviews ( Journal of Medical Virology and Clinica Chimica Acta ). Three systematic reviews in our study were published as letters; one was labeled as a scoping review and another as a rapid review (Table 2 ).

All reviews were published in English, in first quartile (Q1) journals, with JIF ranging from 1.692 to 6.062. One review was empty, meaning that its search did not identify any relevant studies; i.e., no primary studies were included [ 36 ]. The remaining 17 reviews included 269 unique studies; the majority ( N = 211; 78%) were included in only a single review included in our study (range: 1 to 12). Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts, and other observational studies. We found only one review that included randomized clinical trials [ 38 ]. In the included reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020. Ten systematic reviews included meta-analyses. The list of primary studies found in the included systematic reviews is shown in Additional file 4 , as well as the number of reviews in which each primary study was included.

Population and study designs

Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS), or any other correlated complication. One review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [ 36 ], one review was focused on pediatric patients [ 34 ], and one review investigated COVID-19 in pregnant women [ 37 ]. Most reviews assessed clinical symptoms, laboratory findings, or radiological results.

Systematic review findings

The summary of findings from individual reviews is shown in Table 2 . Overall, all-cause mortality ranged from 0.3 to 13.9% (Fig. 2 ).

A meta-analysis of the prevalence of mortality

Clinical symptoms

Seven reviews described the main clinical manifestations of COVID-19 [ 26 , 28 , 29 , 34 , 35 , 39 , 41 ]. Three of them provided only a narrative discussion of symptoms [ 26 , 34 , 35 ]. In the reviews that performed a statistical analysis of the incidence of different clinical symptoms, symptoms in patients with COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figs. 3 , 4 , 5 , 6 , 7 , 8 and 9 ). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).

A meta-analysis of the prevalence of fever

A meta-analysis of the prevalence of cough

A meta-analysis of the prevalence of dyspnea

A meta-analysis of the prevalence of fatigue or myalgia

A meta-analysis of the prevalence of headache

A meta-analysis of the prevalence of gastrointestinal disorders

A meta-analysis of the prevalence of sore throat

Diagnostic aspects

Three reviews described methodologies, protocols, and tools used for establishing the diagnosis of COVID-19 [ 26 , 34 , 38 ]. The use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic method mentioned in the included studies. These diagnostic tests have been widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study with clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least two symptoms suggestive of COVID-19, together with laboratory and chest radiography abnormalities) [ 34 ].

Therapeutic possibilities

Pharmacological and non-pharmacological interventions (supportive therapies) used in treating patients with COVID-19 were reported in five reviews [ 25 , 27 , 34 , 35 , 38 ]. Antivirals used empirically for COVID-19 treatment were reported in seven reviews [ 25 , 27 , 34 , 35 , 37 , 38 , 41 ]; most commonly used were protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analogs (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in two studies [ 25 , 35 ]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation) [ 25 ]. The use of antibiotics, both empirically and to treat secondary pneumonia, was reported in six studies [ 25 , 26 , 27 , 34 , 35 , 38 ]. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [ 27 ]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. The use of mesenchymal stem cells, antifungals, and glucocorticoids were described in four reviews [ 25 , 34 , 35 , 38 ].

Laboratory and radiological findings

Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [ 25 , 29 , 30 , 32 , 33 , 34 , 35 , 39 ]; elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described in those eight reviews. Lippi et al. assessed cardiac troponin I (cTnI) [ 25 ], procalcitonin [ 32 ], and platelet count [ 33 ] in COVID-19 patients. Elevated levels of procalcitonin [ 32 ] and cTnI [ 30 ] were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections [ 33 ].

Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [ 25 , 34 , 35 , 39 , 40 , 41 ]. Those six reviews showed that septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs, and pneumothorax were observed in patients suffering from COVID-19.

Quality of evidence in individual systematic reviews

Table 3 shows the detailed results of the quality assessment of 18 systematic reviews, including the assessment of individual items and summary assessment. A detailed explanation for each decision in each review is available in Additional file 5 .

Using AMSTAR 2 criteria, confidence in the results of all 18 reviews was rated as “critically low” (Table 3 ). Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of an explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).

Risk of bias assessment, based on a reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method, were reported only in one review [ 25 ]. Five reviews presented a table summarizing bias, using various risk of bias tools [ 25 , 29 , 39 , 40 , 41 ]. One review analyzed “study quality” [ 37 ]. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis [ 28 ].

This overview of systematic reviews analyzed the first 18 systematic reviews published after the onset of the COVID-19 pandemic, up to March 24, 2020, with primary studies involving more than 60,000 patients. Using AMSTAR-2, we judged that our confidence in all those reviews was “critically low”. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings, and interventions. We found no systematic reviews on the utility of diagnostic tests.

Symptoms were reported in seven reviews; most of the patients had a fever, cough, dyspnea, myalgia or muscle fatigue, and gastrointestinal disorders such as diarrhea, nausea, or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been described in patients infected with COVID-19 [ 43 ]; however, this was not reported in any of the reviews included in this overview. During the SARS outbreak in 2002, there were reports of impairment of the sense of smell associated with the disease [ 44 , 45 ].

The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control, and treatment facilities; variability in the mortality rate fits the pattern of emerging infectious diseases [ 46 ]. Furthermore, the reported cases did not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries had limited access to diagnostic tests or have implemented testing policies later than the others. Considering the lack of reviews assessing diagnostic testing (sensitivity, specificity, and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, considerable imprecision around the calculated mortality rates existed in the early stage of the COVID-19 pandemic.

Few reviews included treatment data. Those reviews described studies considered to be at a very low level of evidence: usually small, retrospective studies with very heterogeneous populations. Seven reviews analyzed laboratory parameters; those reviews could have been useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, such as assessing which patients need to be reassessed more frequently.

All systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Most of the original studies included in the reviews were case series and case reports, impacting the quality of evidence. Such evidence has major implications for clinical practice and the use of these reviews in evidence-based practice and policy. Clinicians, patients, and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.

We acknowledge that there are numerous challenges associated with analyzing COVID-19 data during a pandemic [ 47 ]. High-quality evidence syntheses are needed for decision-making, but each type of evidence syntheses is associated with its inherent challenges.

The creation of classic systematic reviews requires considerable time and effort; with massive research output, they quickly become outdated, and preparing updated versions also requires considerable time. A recent study showed that updates of non-Cochrane systematic reviews are published a median of 5 years after the publication of the previous version [ 48 ].

Authors may register a review and then abandon it [ 49 ], but the existence of a public record that is not updated may lead other authors to believe that the review is still ongoing. A quarter of Cochrane review protocols remains unpublished as completed systematic reviews 8 years after protocol publication [ 50 ].

Rapid reviews can be used to summarize the evidence, but they involve methodological sacrifices and simplifications to produce information promptly, with inconsistent methodological approaches [ 51 ]. However, rapid reviews are justified in times of public health emergencies, and even Cochrane has resorted to publishing rapid reviews in response to the COVID-19 crisis [ 52 ]. Rapid reviews were eligible for inclusion in this overview, but only one of the 18 reviews included in this study was labeled as a rapid review.

Ideally, COVID-19 evidence would be continually summarized in a series of high-quality living systematic reviews, types of evidence synthesis defined as “ a systematic review which is continually updated, incorporating relevant new evidence as it becomes available ” [ 53 ]. However, conducting living systematic reviews requires considerable resources, calling into question the sustainability of such evidence synthesis over long periods [ 54 ].

Research reports about COVID-19 will contribute to research waste if they are poorly designed, poorly reported, or simply not necessary. In principle, systematic reviews should help reduce research waste as they usually provide recommendations for further research that is needed or may advise that sufficient evidence exists on a particular topic [ 55 ]. However, systematic reviews can also contribute to growing research waste when they are not needed, or poorly conducted and reported. Our present study clearly shows that most of the systematic reviews that were published early on in the COVID-19 pandemic could be categorized as research waste, as our confidence in their results is critically low.

Our study has some limitations. One is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. In three reviews, the methodological quality appraisal was challenging because they were published as letters, or labeled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as “not reported”, resulting in low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low-quality review methods. To make their review available more rapidly and more concisely, the authors may have omitted methodological details. A general issue during a crisis is that speed and completeness must be balanced. However, maintaining high standards requires proper resourcing and commitment to ensure that the users of systematic reviews can have high confidence in the results.

Furthermore, we used adjusted AMSTAR 2 scoring, as the tool was designed for critical appraisal of reviews of interventions. Some reviews may have received lower scores than actually warranted in spite of these adjustments.

Another limitation of our study may be the inclusion of multiple overlapping reviews, as some included reviews included the same primary studies. According to the Cochrane Handbook, including overlapping reviews may be appropriate when the review’s aim is “ to present and describe the current body of systematic review evidence on a topic ” [ 12 ], which was our aim. To avoid bias with summarizing evidence from overlapping reviews, we presented the forest plots without summary estimates. The forest plots serve to inform readers about the effect sizes for outcomes that were reported in each review.

Several authors from this study have contributed to one of the reviews identified [ 25 ]. To reduce the risk of any bias, two authors who did not co-author the review in question initially assessed its quality and limitations.

Finally, we note that the systematic reviews included in our overview may have had issues that our analysis did not identify because we did not analyze their primary studies to verify the accuracy of the data and information they presented. We give two examples to substantiate this possibility. Lovato et al. wrote a commentary on the review of Sun et al. [ 41 ], in which they criticized the authors’ conclusion that sore throat is rare in COVID-19 patients [ 56 ]. Lovato et al. highlighted that multiple studies included in Sun et al. did not accurately describe participants’ clinical presentations, warning that only three studies clearly reported data on sore throat [ 56 ].

In another example, Leung [ 57 ] warned about the review of Li, L.Q. et al. [ 29 ]: “ it is possible that this statistic was computed using overlapped samples, therefore some patients were double counted ”. Li et al. responded to Leung that it is uncertain whether the data overlapped, as they used data from published articles and did not have access to the original data; they also reported that they requested original data and that they plan to re-do their analyses once they receive them; they also urged readers to treat the data with caution [ 58 ]. This points to the evolving nature of evidence during a crisis.

Our study’s strength is that this overview adds to the current knowledge by providing a comprehensive summary of all the evidence synthesis about COVID-19 available early after the onset of the pandemic. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews.

In conclusion, in this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all the reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic could be categorized as research waste. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards to provide patients, clinicians, and decision-makers trustworthy evidence.

Availability of data and materials

All data collected and analyzed within this study are available from the corresponding author on reasonable request.

World Health Organization. Timeline - COVID-19: Available at: https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 1 June 2021.

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available at: https://coronavirus.jhu.edu/map.html . Accessed 1 June 2021.

Anzai A, Kobayashi T, Linton NM, Kinoshita R, Hayashi K, Suzuki A, et al. Assessing the Impact of Reduced Travel on Exportation Dynamics of Novel Coronavirus Infection (COVID-19). J Clin Med. 2020;9(2):601.

Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020;368(6489):395–400. https://doi.org/10.1126/science.aba9757 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fidahic M, Nujic D, Runjic R, Civljak M, Markotic F, Lovric Makaric Z, et al. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med Res Methodol. 2020;20(1):161. https://doi.org/10.1186/s12874-020-01047-2 .

EPPI Centre . COVID-19: a living systematic map of the evidence. Available at: http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx . Accessed 1 June 2021.

NCBI SARS-CoV-2 Resources. Available at: https://www.ncbi.nlm.nih.gov/sars-cov-2/ . Accessed 1 June 2021.

Gustot T. Quality and reproducibility during the COVID-19 pandemic. JHEP Rep. 2020;2(4):100141. https://doi.org/10.1016/j.jhepr.2020.100141 .

Article   PubMed   PubMed Central   Google Scholar  

Kodvanj, I., et al., Publishing of COVID-19 Preprints in Peer-reviewed Journals, Preprinting Trends, Public Discussion and Quality Issues. Preprint article. bioRxiv 2020.11.23.394577; doi: https://doi.org/10.1101/2020.11.23.394577 .

Dobler CC. Poor quality research and clinical practice during COVID-19. Breathe (Sheff). 2020;16(2):200112. https://doi.org/10.1183/20734735.0112-2020 .

Article   Google Scholar  

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 .

Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/10.1186/s13643-017-0617-1 .

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. 2020. Available from www.training.cochrane.org/handbook .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.1 (updated September 2020). Cochrane. 2020; Available from www.training.cochrane.org/handbook .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18. https://doi.org/10.1186/s13643-018-0914-3 .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):29. https://doi.org/10.1186/s13643-018-0768-8 .

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/10.1186/s13643-018-0695-8 .

Pollock M, Fernandes RM, Pieper D, Tricco AC, Gates M, Gates A, et al. Preferred reporting items for overviews of reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):335. https://doi.org/10.1186/s13643-019-1252-9 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0 .

Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5. https://doi.org/10.1016/j.jclinepi.2017.08.002 .

Article   PubMed   Google Scholar  

Gates M, Gates A, Guitard S, Pollock M, Hartling L. Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review. Syst Rev. 2020;9(1):254. https://doi.org/10.1186/s13643-020-01509-0 .

Covidence - systematic review software. Available at: https://www.covidence.org/ . Accessed 1 June 2021.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Borges do Nascimento IJ, et al. Novel Coronavirus Infection (COVID-19) in Humans: A Scoping Review and Meta-Analysis. J Clin Med. 2020;9(4):941.

Article   PubMed Central   Google Scholar  

Adhikari SP, Meng S, Wu YJ, Mao YP, Ye RX, Wang QZ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty. 2020;9(1):29. https://doi.org/10.1186/s40249-020-00646-x .

Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020;57:279–83. https://doi.org/10.1016/j.jcrc.2020.03.005 .

Li B, Yang J, Zhao F, Zhi L, Wang X, Liu L, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol. 2020;109(5):531–8. https://doi.org/10.1007/s00392-020-01626-9 .

Article   CAS   PubMed   Google Scholar  

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(6):577–83. https://doi.org/10.1002/jmv.25757 .

Lippi G, Lavie CJ, Sanchis-Gomar F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): evidence from a meta-analysis. Prog Cardiovasc Dis. 2020;63(3):390–1. https://doi.org/10.1016/j.pcad.2020.03.001 .

Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107–8. https://doi.org/10.1016/j.ejim.2020.03.014 .

Lippi G, Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–1. https://doi.org/10.1016/j.cca.2020.03.004 .

Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145–8. https://doi.org/10.1016/j.cca.2020.03.022 .

Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270 .

Lupia T, Scabini S, Mornese Pinna S, di Perri G, de Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: a new challenge. J Glob Antimicrob Resist. 2020;21:22–7. https://doi.org/10.1016/j.jgar.2020.02.021 .

Marasinghe, K.M., A systematic review investigating the effectiveness of face mask use in limiting the spread of COVID-19 among medically not diagnosed individuals: shedding light on current recommendations provided to individuals not medically diagnosed with COVID-19. Research Square. Preprint article. doi : https://doi.org/10.21203/rs.3.rs-16701/v1 . 2020 .

Mullins E, Evans D, Viner RM, O’Brien P, Morris E. Coronavirus in pregnancy and delivery: rapid review. Ultrasound Obstet Gynecol. 2020;55(5):586–92. https://doi.org/10.1002/uog.22014 .

Pang J, Wang MX, Ang IYH, Tan SHX, Lewis RF, Chen JIP, et al. Potential Rapid Diagnostics, Vaccine and Therapeutics for 2019 Novel coronavirus (2019-nCoV): a systematic review. J Clin Med. 2020;9(3):623.

Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, Villamizar-Peña R, Holguin-Rivera Y, Escalera-Antezana JP, et al. Clinical, laboratory and imaging features of COVID-19: a systematic review and meta-analysis. Travel Med Infect Dis. 2020;34:101623. https://doi.org/10.1016/j.tmaid.2020.101623 .

Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. https://doi.org/10.2214/AJR.20.23034 .

Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612–7. https://doi.org/10.1002/jmv.25735 .

Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis. 2020;94:91–5. https://doi.org/10.1016/j.ijid.2020.03.017 .

Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: challenges for fighting the storm. Eur J Clin Investig. 2020;50(3):e13209. https://doi.org/10.1111/eci.13209 .

Article   CAS   Google Scholar  

Hwang CS. Olfactory neuropathy in severe acute respiratory syndrome: report of a case. Acta Neurol Taiwanica. 2006;15(1):26–8.

Google Scholar  

Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, et al. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272–7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e .

Rajgor DD, Lee MH, Archuleta S, Bagdasarian N, Quek SC. The many estimates of the COVID-19 case fatality rate. Lancet Infect Dis. 2020;20(7):776–7. https://doi.org/10.1016/S1473-3099(20)30244-9 .

Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol. 2020;20(1):81. https://doi.org/10.1186/s12874-020-00972-6 .

Rombey T, Lochner V, Puljak L, Könsgen N, Mathes T, Pieper D. Epidemiology and reporting characteristics of non-Cochrane updates of systematic reviews: a cross-sectional study. Res Synth Methods. 2020;11(3):471–83. https://doi.org/10.1002/jrsm.1409 .

Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114–21. https://doi.org/10.1016/j.jclinepi.2019.08.010 .

Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13(1):224. https://doi.org/10.1186/s12916-015-0465-6 .

COVID-19 Rapid Reviews: Cochrane’s response so far. Available at: https://training.cochrane.org/resource/covid-19-rapid-reviews-cochrane-response-so-far . Accessed 1 June 2021.

Cochrane. Living systematic reviews. Available at: https://community.cochrane.org/review-production/production-resources/living-systematic-reviews . Accessed 1 June 2021.

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T. Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019;8(1):325. https://doi.org/10.1186/s13643-019-1248-5 .

Babic A, Poklepovic Pericic T, Pieper D, Puljak L. How to decide whether a systematic review is stable and not in need of updating: analysis of Cochrane reviews. Res Synth Methods. 2020;11(6):884–90. https://doi.org/10.1002/jrsm.1451 .

Lovato A, Rossettini G, de Filippis C. Sore throat in COVID-19: comment on “clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis”. J Med Virol. 2020;92(7):714–5. https://doi.org/10.1002/jmv.25815 .

Leung C. Comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1431–2. https://doi.org/10.1002/jmv.25912 .

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. Response to Char’s comment: comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1433. https://doi.org/10.1002/jmv.25924 .

Download references

Acknowledgments

We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from the Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Additional file 6 . We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.

This research received no external funding.

Author information

Authors and affiliations.

University Hospital and School of Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

Israel Júnior Borges do Nascimento & Milena Soriano Marcolino

Medical College of Wisconsin, Milwaukee, WI, USA

Israel Júnior Borges do Nascimento

Helene Fuld Health Trust National Institute for Evidence-based Practice in Nursing and Healthcare, College of Nursing, The Ohio State University, Columbus, OH, USA

Dónal P. O’Mathúna

School of Nursing, Psychotherapy and Community Health, Dublin City University, Dublin, Ireland

Department of Anesthesiology, Intensive Care and Pain Medicine, University of Münster, Münster, Germany

Thilo Caspar von Groote

Department of Sport and Health Science, Technische Universität München, Munich, Germany

Hebatullah Mohamed Abdulazeem

School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Callaghan, Australia

Ishanka Weerasekara

Department of Physiotherapy, Faculty of Allied Health Sciences, University of Peradeniya, Peradeniya, Sri Lanka

Cochrane Croatia, University of Split, School of Medicine, Split, Croatia

Ana Marusic, Irena Zakarija-Grkovic & Tina Poklepovic Pericic

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Cochrane Brazil, Evidence-Based Health Program, Universidade Federal de São Paulo, São Paulo, Brazil

Vinicius Tassoni Civile & Alvaro Nagib Atallah

Yorkville University, Fredericton, New Brunswick, Canada

Santino Filoso

Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada

Nicola Luigi Bragazzi

You can also search for this author in PubMed   Google Scholar

Contributions

IJBN conceived the research idea and worked as a project coordinator. DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP, ANA, SF, NLB and MSM were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. All authors revised the manuscript critically for the content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

Not required as data was based on published studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1..

Search strategies used in the study.

Additional file 2: Appendix 2.

Adjusted scoring of AMSTAR 2 used in this study for systematic reviews of studies that did not analyze interventions.

Additional file 3: Appendix 3.

List of excluded studies, with reasons.

Additional file 4: Appendix 4.

Table of overlapping studies, containing the list of primary studies included, their visual overlap in individual systematic reviews, and the number in how many reviews each primary study was included.

Additional file 5: Appendix 5.

A detailed explanation of AMSTAR scoring for each item in each review.

Additional file 6: Appendix 6.

List of members and affiliates of International Network of Coronavirus Disease 2019 (InterNetCOVID-19).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Borges do Nascimento, I.J., O’Mathúna, D.P., von Groote, T.C. et al. Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews. BMC Infect Dis 21 , 525 (2021). https://doi.org/10.1186/s12879-021-06214-4

Download citation

Received : 12 April 2020

Accepted : 19 May 2021

Published : 04 June 2021

DOI : https://doi.org/10.1186/s12879-021-06214-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus
• Evidence-based medicine
• Infectious diseases

BMC Infectious Diseases

ISSN: 1471-2334

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

The impact of the COVID-19 pandemic on scientific research in the life sciences

Massimo riccaboni.

1 AXES, IMT School for Advanced Studies Lucca, Lucca, Italy

Luca Verginer

2 Chair of Systems Design D-MTEC, ETH Zürich, Zurich, Switzerland

Associated Data

The processed data, instructions on how to process the raw PubMed dataset as well as all code are available via Zenodo at https://doi.org/10.5281/zenodo.5121216 .

The COVID-19 outbreak has posed an unprecedented challenge to humanity and science. On the one side, public and private incentives have been put in place to promptly allocate resources toward research areas strictly related to the COVID-19 emergency. However, research in many fields not directly related to the pandemic has been displaced. In this paper, we assess the impact of COVID-19 on world scientific production in the life sciences and find indications that the usage of medical subject headings (MeSH) has changed following the outbreak. We estimate through a difference-in-differences approach the impact of the start of the COVID-19 pandemic on scientific production using the PubMed database (3.6 Million research papers). We find that COVID-19-related MeSH terms have experienced a 6.5 fold increase in output on average, while publications on unrelated MeSH terms dropped by 10 to 12%. The publication weighted impact has an even more pronounced negative effect (-16% to -19%). Moreover, COVID-19 has displaced clinical trial publications (-24%) and diverted grants from research areas not closely related to COVID-19. Note that since COVID-19 publications may have been fast-tracked, the sudden surge in COVID-19 publications might be driven by editorial policy.

Introduction

The COVID-19 pandemic has mobilized the world scientific community in 2020, especially in the life sciences [ 1 , 2 ]. In the first three months after the pandemic, the number of scientific papers about COVID-19 was fivefold the number of articles on H1N1 swine influenza [ 3 ]. Similarly, the number of clinical trials related to COVID-19 prophylaxis and treatments skyrocketed [ 4 ]. Thanks to the rapid mobilization of the world scientific community, COVID-19 vaccines have been developed in record time. Despite this undeniable success, there is a rising concern about the negative consequences of COVID-19 on clinical trial research, with many projects being postponed [ 5 – 7 ]. According to Evaluate Pharma, clinical trials were one of the pandemic’s first casualties, with a record number of 160 studies suspended for reasons related to COVID-19 in April 2020 [ 8 , 9 ] reporting a total of 1,200 trials suspended as of July 2020. As a consequence, clinical researchers have been impaired by reduced access to healthcare research infrastructures. Particularly, the COVID-19 outbreak took a tall on women and early-career scientists [ 10 – 13 ]. On a different ground, Shan and colleagues found that non-COVID-19-related articles decreased as COVID-19-related articles increased in top clinical research journals [ 14 ]. Fraser and coworker found that COVID-19 preprints received more attention and citations than non-COVID-19 preprints [ 1 ]. More recently, Hook and Porter have found some early evidence of ‘covidisation’ of academic research, with research grants and output diverted to COVID-19 research in 2020 [ 15 ]. How much should scientists switch their efforts toward SARS-CoV-2 prevention, treatment, or mitigation? There is a growing consensus that the current level of ‘covidisation’ of research can be wasteful [ 4 , 5 , 16 ].

Against this background, in this paper, we investigate if the COVID-19 pandemic has induced a shift in biomedical publications toward COVID-19-related scientific production. The objective of the study is to show that scientific articles listing covid-related Medical Subject Headings (MeSH) when compared against covid-unrelated MeSH have been partially displaced. Specifically, we look at several indicators of scientific production in the life sciences before and after the start of the COVID-19 pandemic: (1) number of papers published, (2) impact factor weighted number of papers, (3) opens access, (4) number of publications related to clinical trials, (5) number of papers listing grants, (6) number of papers listing grants existing before the pandemic. Through a natural experiment approach, we analyze the impact of the pandemic on scientific production in the life sciences. We consider COVID-19 an unexpected and unprecedented exogenous source of variation with heterogeneous effects across biomedical research fields (i.e., MeSH terms).

Based on the difference in difference results, we document the displacement effect that the pandemic has had on several aspects of scientific publishing. The overall picture that emerges from this analysis is that there has been a profound realignment of priorities and research efforts. This shift has displaced biomedical research in fields not related to COVID-19.

The rest of the paper is structured as follows. First, we describe the data and our measure of relatedness to COVID-19. Next, we illustrate the difference-in-differences specification we rely on to identify the impact of the pandemic on scientific output. In the results section, we present the results of the difference-in-differences and network analyses. We document the sudden shift in publications, grants and trials towards COVID-19-related MeSH terms. Finally, we discuss the findings and highlight several policy implications.

Materials and methods

The present analysis is based primarily on PubMed and the Medical Subject Headings (MeSH) terminology. This data is used to estimate the effect of the start of the COVID 19 pandemic via a difference in difference approach. This section is structured as follows. We first introduce the data and then the econometric methodology. This analysis is not based on a pre-registered protocol.

Selection of biomedical publications

We rely on PubMed, a repository with more than 34 million biomedical citations, for the analysis. Specifically, we analyze the daily updated files up to 31/06/2021, extracting all publications of type ‘Journal Article’. For the principal analysis, we consider 3,638,584 papers published from January 2019 to December 2020. We also analyze 11,122,017 papers published from 2010 onwards to identify the earliest usage of a grant and infer if it was new in 2020. We use the SCImago journal ranking statistics to compute the impact factor weighted number (IFWN) of papers in a given field of research. To assign the publication date, we use the ‘electronically published’ dates and, if missing, the ‘print published’ dates.

Medical subject headings

We rely on the Medical Subject Headings (MeSH) terminology to approximate narrowly defined biomedical research fields. This terminology is a curated medical vocabulary, which is manually added to papers in the PubMed corpus. The fact that MeSH terms are manually annotated makes this terminology ideal for classification purposes. However, there is a delay between publication and annotation, on the order of several months. To address this delay and have the most recent classification, we search for all 28 425 MeSH terms using PubMed’s ESearch utility and classify paper by the results. The specific API endpoint is https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi , the relevant scripts are available with the code. For example, we assign the term ‘Ageusia’ (MeSH ID D000370) to all papers listed in the results of the ESearch API. We apply this method to the whole period (January 2019—December 2020) and obtain a mapping from papers to the MeSH terms. For every MeSH term, we keep track of the year they have been established. For instance, COVID-19 terms were established in 2020 (see Table 1 ): in January 2020, the WHO recommended 2019-nCoV and 2019-nCoV acute respiratory disease as provisional names for the virus and disease. The WHO issued the official terms COVID-19 and SARS-CoV-2 at the beginning of February 2020. By manually annotating publications, all publications referring to COVID-19 and SARS-CoV-2 since January 2020 have been labelled with the related MeSH terms. Other MeSH terms related to COVID-19, such as coronavirus, for instance, have been established years before the pandemic (see Table 2 ). We proxy MeSH term usage via search terms using the PubMed EUtilities API; this means that we are not using the hand-labelled MeSH terms but rather the PubMed search results. This means that the accuracy of the MeSH term we assign to a given paper is not perfect. In practice, this means that we have assigned more MeSH terms to a given term than a human annotator would have.

The list contains only terms with at least 100 publications in 2020.

Clinical trials and publication types

We classify publications using PubMed’s ‘PublicationType’ field in the XML baseline files (There are 187 publication types, see https://www.nlm.nih.gov/mesh/pubtypes.html ). We consider a publication to be related to a clinical trial if it lists any of the following descriptors:

• D016430: Clinical Trial
• D017426: Clinical Trial, Phase I
• D017427: Clinical Trial, Phase II
• D017428: Clinical Trial, Phase III
• D017429: Clinical Trial, Phase IV
• D018848: Controlled Clinical Trial
• D065007: Pragmatic Clinical Trial
• D000076362: Adaptive Clinical Trial
• D000077522: Clinical Trial, Veterinary

In our analysis of the impact of COVID-19 on publications related to clinical trials, we only consider MeSH terms that are associated at least once with a clinical trial publication over the two years. We apply this restriction to filter out MeSH terms that are very unlikely to be relevant for clinical trial types of research.

Open access

We proxy the availability of a journal article to the public, i.e., open access, if it is available from PubMed Central. PubMed Central archives full-text journal articles and provides free access to the public. Note that the copyright license may vary across participating publishers. However, the text of the paper is for all effects and purposes freely available without requiring subscriptions or special affiliation.

We infer if a publication has been funded by checking if it lists any grants. We classify grants as either ‘old’, i.e. existed before 2019, or ‘new’, i.e. first observed afterwards. To do so, we collect all grant IDs for 11,122,017 papers from 2010 on-wards and record their first appearance. This procedure is an indirect inference of the year the grant has been granted. The basic assumption is that if a grant number has not been listed in any publication since 2010, it is very likely a new grant. Specifically, an old grant is a grant listed since 2019 observed at least once from 2010 to 2018.

Note that this procedure is only approximate and has a few shortcomings. Mistyped grant numbers (e.g. ‘1234-M JPN’ and ‘1234-M-JPN’) could appear as new grants, even though they existed before, or new grants might be classified as old grants if they have a common ID (e.g. ‘Grant 1’). Unfortunately, there is no central repository of grant numbers and the associated metadata; however, there are plans to assign DOI numbers to grants to alleviate this problem (See https://gitlab.com/crossref/open_funder_registry for the project).

Impact factor weighted publication numbers (IFWN)

In our analysis, we consider two measures of scientific output. First, we simply count the number of publications by MeSH term. However, since journals vary considerably in terms of impact factor, we also weigh the number of publications by the impact factor of the venue (e.g., journal) where it was published. Specifically, we use the SCImago journal ranking statistics to weigh a paper by the impact factor of the journal it appears in. We use the ‘citation per document in the past two years’ for 45,230 ISSNs. Note that a journal may and often has more than one ISSN, i.e., one for the printed edition and one for the online edition. SCImago applies the same score for a venue across linked ISSNs.

For the impact factor weighted number (IFWN) of publication per MeSH terms, this means that all publications are replaced by the impact score of the journal they appear in and summed up.

COVID-19-relatedness

To measure how closely related to COVID-19 is a MeSH term, we introduce an index of relatedness to COVID-19. First, we identify the focal COVID-19 terms, which appeared in the literature in 2020 (see Table 1 ). Next, for all other pre-existing MeSH terms, we measure how closely related to COVID-19 they end up being.

Our aim is to show that MeSH terms that existed before and are related have experienced a sudden increase in the number of (impact factor weighted) papers.

We define a MeSH term’s COVID-19 relatedness as the conditional probability that, given its appearance on a paper, also one of the focal COVID-19 terms listed in Table 1 are present. In other words, the relatedness of a MeSH term is given by the probability that a COVID-19 MeSH term appears alongside. Since the focal COVID-19 terms did not exist before 2020, we estimate this measure only using papers published since January 2020. Formally, we define COVID-19-relatedness ( σ ) as in Eq (1) , where C is the set of papers listing a COVID-19 MeSH term and M i is the set of papers listing MeSH term i .

Intuitively we can read this measure as: what is the probability in 2020 that a COVID-19 MeSH term is present given that we chose a paper with MeSH term i ? For example, given that in 2020 we choose a paper dealing with “Ageusia” (i.e., Complete or severe loss of the subjective sense of taste), there is a 96% probability that this paper also lists COVID-19, see Table 1 .

Note that a paper listing a related MeSH term does not imply that that paper is doing COVID-19 research, but it implies that one of the MeSH terms listed is often used in COVID-19 research.

In sum, in our analysis, we use the following variables:

• Papers: Number of papers by MeSH term;
• Impact: Impact factor weighted number of papers by MeSH term;
• PMC: Papers listed in PubMed central by MeSH term, as a measure of Open Access publications;
• Trials: number of publications of type “Clinical Trial” by MeSH term;
• Grants: number of papers with at least one grant by MeSH term;
• Old Grants: number of papers listing a grant that has been observed between 2010 and 2018, by MeSH term;

Difference-in-differences

The difference-in-differences (DiD) method is an econometric technique to imitate an experimental research design from observation data, sometimes referred to as a quasi-experimental setup. In a randomized controlled trial, subjects are randomly assigned either to the treated or the control group. Analogously, in this natural experiment, we assume that medical subject headings (MeSH) have been randomly assigned to be either treated (related) or not treated (unrelated) by the pandemic crisis.

Before the COVID, for a future health crisis, the set of potentially impacted medical knowledge was not predictable since it depended on the specifics of the emergency. For instance, ageusia (loss of taste), a medical concept existing since 1991, became known to be a specific symptom of COVID-19 only after the pandemic.

Specifically, we exploit the COVID-19 as an unpredictable and exogenous shock that has deeply affected the publication priorities for biomedical scientific production, as compared to the situation before the pandemic. In this setting, COVID-19 is the treatment, and the identification of this new human coronavirus is the event. We claim that treated MeSH terms, i.e., MeSH terms related to COVID-19, have experienced a sudden increase in terms of scientific production and attention. In contrast, research on untreated MeSH terms, i.e., MeSH terms not related to COVID-19, has been displaced by COVID-19. Our analysis compares the scientific output of COVID-19 related and unrelated MeSH terms before and after January 2020.

Consider the simple regression model in Eq (2) . We have an outcome Y and dummy variable P identifying the period as before the event P = 0 and P = 1 as after the event. Additionally, we have a dummy variable identifying an observation belonging to the treated group ( G = 1) or control ( G = 0) group.

In our case, some of the terms turn out to be related to COVID-19 in 2020, whereas most of the MeSH terms are not closely related to COVID-19.

Thus β 1 identifies the overall effect on the control group after the event, β 2 the difference across treated and control groups before the event (i.e. the first difference in DiD) and finally the effect on the treated group after the event, net of the first difference, β 3 . This last parameter identifies the treatment effect on the treated group netting out the pre-treatment difference.

For the DiD to have a causal interpretation, it must be noted that pre-event, the trends of the two groups should be parallel, i.e., the common trend assumption (CTA) must be satisfied. We will show that the CTA holds in the results section.

To specify the DiD model, we need to define a period before and after the event and assign a treatment status or level of exposure to each term.

Before and after

The pre-treatment period is defined as January 2019 to December 2019. The post-treatment period is defined as the months from January 2020 to December 2020. We argue that the state of biomedical research was similar in those two years, apart from the effect of the pandemic.

Treatment status and exposure

The treatment is determined by the COVID-19 relatedness index σ i introduced earlier. Specifically, this number indicates the likelihood that COVID-19 will be a listed MeSH term, given that we observe the focal MeSH term i . To show that the effect becomes even stronger the closer related the subject is, and for ease of interpretation, we also discretize the relatedness value into three levels of treatment. Namely, we group MeSH terms with a σ between, 0% to 20%, 20% to 80% and 80% to 100%. The choice of alternative grouping strategies does not significantly affect our results. Results for alternative thresholds of relatedness can be computed using the available source code. We complement the dichotomized analysis by using the treatment intensity (relatedness measure σ ) to show that the result persists.

Panel regression

In this work, we estimate a random effects panel regression where the units of analysis are 28 318 biomedical research fields (i.e. MeSH terms) observed over time before and after the COVID-19 pandemic. The time resolution is at the monthly level, meaning that for each MeSH term, we have 24 observations from January 2019 to December 2020.

The basic panel regression with continuous treatment follows a similar setup as Eq (2) but with MeSH term random effects and monthly fixed effects.

The outcome variable Y it identifies the outcome at time t (i.e., month), for MeSH term i . As before, P t identifies the period with P t = 0 if the month is before January 2020 and P t = 1 if it is on or after this date. In (3) , the treatment level is measure by the relatedness to COVID-19 ( σ i ), where again the γ 1 identifies pre-trend (constant) differences and δ 1 the overall effect.

As mentioned before, to highlight that the effect is not linear but increases with relatedness, we split σ into three groups: from 0% to 20%, 20% to 80% and 80% to 100%. In the three-level treatment specification, the number of treatment levels ( G i ) is 3; hence we have two γ parameters. Note that I (⋅) is the indicator function, which is 1 if the argument is true, and 0 otherwise.

In total, we estimate six coefficients. As before, the δ l coefficient identifies the DiD effect.

Verifying the Common Trend Assumption (CTA)

To show that the pre-event trends are parallel and that the effect on publication activity is only visible from January 2020, we estimate a panel regression with each month modelled as a different event. Specifically, we estimate the following model.

We show that the CTA holds for this model by comparing the pre-event trends of the control group to the treated groups (COVID-19 related MeSH terms). Namely, we show that the pre-event trends of the control group are the same as the pre-event trends of the treated group.

Co-occurrence analysis

To investigate if the pandemic has caused a reconfiguration of research priorities, we look at the MeSH term co-occurrence network. Precisely, we extract the co-occurrence network of all 28,318 MeSH terms as they appear in the 3.3 million papers. We considered the co-occurrence networks of 2018, 2019 and 2020. Each node represents a MeSH term in these networks, and a link between them indicates that they have been observed at least once together. The weight of the edge between the MeSH terms is given by the number of times those terms have been jointly observed in the same publications.

Medical language is hugely complicated, and this simple representation does not capture the intricacies, subtle nuances and, in fact, meaning of the terms. Therefore, we do not claim that we can identify how the actual usage of MeSH terms has changed from this object, but rather that it has. Nevertheless, the co-occurrence graph captures rudimentary relations between concepts. We argue that absent a shock to the system, their basic usage patterns, change in importance (within the network) would essentially be the same from year to year. However, if we find that the importance of terms changes more than expected in 2020, it stands to reason that there have been some significant changes.

To show that that MeSH usage has been affected, we compute for each term in the years 2018, 2019 and 2020 their PageRank centrality [ 17 ]. The PageRank centrality tells us how likely a random walker traversing a network would be found at a given node if she follows the weights of the empirical edges (i.e., co-usage probability). Specifically, for the case of the MeSH co-occurrence network, this number represents how often an annotator at the National Library of Medicine would assign that MeSH term following the observed general usage patterns. It is a simplistic measure to capture the complexities of biomedical research. Nevertheless, it captures far-reaching interdependence across MeSH terms as the measure uses the whole network to determine the centrality of every MeSH term. A sudden change in the rankings and thus the position of MeSH terms in this network suggests that a given research subject has risen as it is used more often with other important MeSH terms (or vice versa).

To show that COVID-19-related research has profoundly impacted the way MeSH terms are used, we compute for each MeSH term the change in its PageRank centrality ( p it ).

We then compare the growth for each MeSH i term in g i (2019), i.e. before the the COVID-19 pandemic, with the growth after the event ( g i (2020)).

Publication growth

To estimate growth in scientific output, we compute the year over year growth in the number of the impact weighted number of publications per MeSH. Specifically, we measure the year by year growth as defined below, where m is the impact weighted number of publications at time t .

Changes in output and COVID-19 relatedness

Before we show the regression results, we provide descriptive evidence that publications from 2019 to 2020 have drastically increased. By showing that this growth correlates strongly with a MeSH term’s COVID-19 relatedness ( σ ), we demonstrate that (1) σ captures an essential aspect of the growth dynamics and (2) highlight the meteoric rise of highly related terms.

We look at the year over year growth in the number of the impact weighted number of publications per MeSH term from 2018 to 2019 and 2019 to 2020 as defined in the methods section.

Fig 1 shows the yearly growth of the impact weighted number of publications per MeSH term. By comparing the growth of the number of publications from the years 2018, 2019 and 2020, we find that the impact factor weighted number of publications has increased by up to a factor of 100 compared to the previous year for Betacoronavirus, one of the most closely related to COVID-19 MeSH term.

Each dot represents, a MeSH term. The y axis (growth) is in symmetric log scale. The x axis shows the COVID-19 relatedness, σ . Note that the position of the dots on the x-axis is the same in the two plots. Below: MeSH term importance gain (PageRank) and their COVID-19 relatedness.

Fig 1 , first row, reveals how strongly correlated the growth in the IFWN of publication is to the term’s COVID-19 relatedness. For instance, we see that the term ‘Betacoronavirus’ skyrocketed from 2019 to 2020, which is expected given that SARS-CoV-2 is a species of the genus. Conversely, the term ‘Alphacoronavirus’ has not experienced any growth given that it is twin a genus of the Coronaviridae family, but SARS-CoV-2 is not one of its species. Note also the fast growth in the number of publications dealing with ‘Quarantine’. Moreover, MeSH terms that grew significantly from 2018 to 2019 and were not closely related to COVID-19, like ‘Vaping’, slowed down in 2020. From the graph, the picture emerges that publication growth is correlated with COVID-19 relatedness σ and that the growth for less related terms slowed down.

To show that the usage pattern of MeSH terms has changed following the pandemic, we compute the PageRank centrality using graph-tool [ 18 ] as discussed in the Methods section.

Fig 1 , second row, shows the change in the PageRank centrality of the MeSH terms after the pandemic (2019 to 2020, right plot) and before (2018 to 2019, left plot). If there were no change in the general usage pattern, we would expect the variance in PageRank changes to be narrow across the two periods, see (left plot). However, PageRank scores changed significantly more from 2019 to 2020 than from 2018 to 2019, suggesting that there has been a reconfiguration of the network.

To further support this argument, we carry out a DiD regression analysis.

Common trends assumption

As discussed in the Methods section, we need to show that the CTA assumption holds for the DiD to be defined appropriately. We do this by estimating for each month the number of publications and comparing it across treatment groups. This exercise also serves the purpose of a placebo test. By assuming that each month could have potentially been the event’s timing (i.e., the outbreak), we show that January 2020 is the most likely timing of the event. The regression table, as noted earlier, contains over 70 estimated coefficients, hence for ease of reading, we will only show the predicted outcome per month by group (see Fig 2 ). The full regression table with all coefficients is available in the S1 Table .

The y axis is in log scale. The dashed vertical line identifies January 2020. The dashed horizontal line shows the publications in January 2019 for the 0–20% group before the event. This line highlights that the drop happens after the event. The bands around the lines indicate the 95% confidence interval of the predicted values. The results are the output of the Stata margins command.

Fig 2 shows the predicted number per outcome variable obtained from the panel regression model. These predictions correspond to the predicted value per relatedness group using the regression parameters estimated via the linear panel regression. The bands around the curves are the 95% confidence intervals.

All outcome measures depict a similar trend per month. Before the event (i.e., January 2020), there is a common trend across all groups. In contrast, after the event, we observe a sudden rise for the outcomes of the COVID-19 related treated groups (green and red lines) and a decline in the outcomes for the unrelated group (blue line). Therefore, we can conclude that the CTA assumption holds.

Regression results

Table 3 shows the DiD regression results (see Eq (3) ) for the selected outcome measures: number of publications (Papers), impact factor weighted number of publications (Impact), open access (OA) publications, clinical trial related publications, and publications with existing grants.

t statistics in parentheses, Std. Err. adjusted by MeSH-id. All outcome variables are in natural log.

* p < 0.05,

** p < 0.01,

*** p < 0.001

Table 3 shows results for the discrete treatment level version of the DiD model (see Eq (4) ).

Note that the outcome variable is in natural log scale; hence to get the effect of the independent variable, we need to exponentiate the coefficient. For values close to 0, the effect is well approximated by the percentage change of that magnitude.

In both specifications we see that the least related group, drops in the number of publications between 10% and 13%, respectively (first row of Tables ​ Tables3 3 and ​ and4, 4 , exp(−0.102) ≈ 0.87). In line with our expectations, the increase in the number of papers published by MeSH term is positively affected by the relatedness to COVID-19. In the discrete model (row 2), we note that the number of documents with MeSH terms with a COVID-19 relatedness between 20 and 80% grows by 18% and highly related terms by a factor of approximately 6.6 (exp(1.88)). The same general pattern can be observed for the impact weighted publication number, i.e., Model (2). Note, however, that the drop in the impact factor weighted output is more significant, reaching -19% for COVID-19 unrelated publications, and related publications growing by a factor of 8.7. This difference suggests that there might be a bias to publish papers on COVID-19 related subjects in high impact factor journals.

t statistics in parentheses, Std. Err. adjusted by MeSH-id. All outcome variables are in natural log. σ is the MeSH term relatedness to COVID-19.

By looking at the number of open access publications (PMC), we note that the least related group has not been affected negatively by the pandemic. However, the number of COVID-19 related publications has drastically increased for the most COVID-19 related group by a factor of 6.2. Note that the substantial increase in the number of papers available through open access is in large part due to journal and editorial policies to make preferentially COVID research immediately available to the public.

Regarding the number of clinical trial publications, we note that the least related group has been affected negatively, with the number of publications on clinical trials dropping by a staggering 24%. At the same time, publications on clinical trials for COVID-19-related MeSH have increased by a factor of 2.1. Note, however, that the effect on clinical trials is not significant in the continuous regression. The discrepancy across Tables ​ Tables3 3 and ​ and4 4 highlights that, especially for trials, the effect is not linear, where only the publications on clinical trials closely related to COVID-19 experiencing a boost.

It has been reported [ 19 ] that while the number of clinical trials registered to treat or prevent COVID-19 has surged with 179 new registrations in the second week of April 2020 alone. Only a few of these have led to publishable results in the 12 months since [ 20 ]. On the other hand, we find that clinical trial publications, considering related MeSH (but not COVID-19 directly), have had significant growth from the beginning of the pandemic. These results are not contradictory. Indeed counting the number of clinical trial publications listing the exact COVID-19 MeSH term (D000086382), we find 212 publications. While this might seem like a small number, consider that in 2020 only 8,485 publications were classified as clinical trials; thus, targeted trials still made up 2.5% of all clinical trials in 2020 . So while one might doubt the effectiveness of these research efforts, it is still the case that by sheer number, they represent a significant proportion of all publications on clinical trials in 2020. Moreover, COVID-19 specific Clinical trial publications in 2020, being a delayed signal of the actual trials, are a lower bound estimate on the true number of such clinical trials being conducted. This is because COVID-19 studies could only have commenced in 2020, whereas other studies had a head start. Thus our reported estimates are conservative, meaning that the true effect on actual clinical trials is likely larger, not smaller.

Research funding, as proxied by the number of publications with grants, follows a similar pattern, but notably, COVID-19-related MeSH terms list the same proportion of grants established before 2019 as other unrelated MeSH terms, suggesting that grants which were not designated for COVID-19 research have been used to support COVID-19 related research. Overall, the number of publications listing a grant has dropped. Note that this should be because the number of publications overall in the unrelated group has dropped. However, we note that the drop in publications is 10% while the decline in publications with at least one grant is 15%. This difference suggests that publications listing grants, which should have more funding, are disproportionately COVID-19 related papers. To further investigate this aspect, we look at whether the grant was old (pre-2019) or appeared for the first time in or after 2019. It stands to reason that an old grant (pre-2019) would not have been granted for a project dealing with the pandemic. Hence we would expect that COVID-19 related MeSH terms to have a lower proportion of old grants than the unrelated group. In models (6) in Table 4 we show that the number of old grants for the unrelated group drops by 13%. At the same time, the number of papers listing old grants (i.e., pre-2019) among the most related group increased by a factor of 3.1. Overall, these results suggest that COVID-19 related research has been funded largely by pre-existing grants, even though a specific mandate tied to the grants for this use is unlikely.

The scientific community has swiftly reallocated research efforts to cope with the COVID-19 pandemic, mobilizing knowledge across disciplines to find innovative solutions in record time. We document this both in terms of changing trends in the biomedical scientific output and the usage of MeSH terms by the scientific community. The flip side of this sudden and energetic prioritization of effort to fight COVID-19 has been a sudden contraction of scientific production in other relevant research areas. All in all, we find strong support to the hypotheses that the COVID-19 crisis has induced a sudden increase of research output in COVID-19 related areas of biomedical research. Conversely, research in areas not related to COVID-19 has experienced a significant drop in overall publishing rates and funding.

Our paper contributes to the literature on the impact of COVID-19 on scientific research: we corroborate previous findings about the surge of COVID-19 related publications [ 1 – 3 ], partially displacing research in COVID-19 unrelated fields of research [ 4 , 14 ], particularly research related to clinical trials [ 5 – 7 ]. The drop in trial research might have severe consequences for patients affected by life-threatening diseases since it will delay access to new and better treatments. We also confirm the impact of COVID-19 on open access publication output [ 1 ]; also, this is milder than traditional outlets. On top of this, we provide more robust evidence on the impact weighted effect of COVID-19 and grant financed research, highlighting the strong displacement effect of COVID-19 on the allocation of financial resources [ 15 ]. We document a substantial change in the usage patterns of MeSH terms, suggesting that there has been a reconfiguration in the way research terms are being combined. MeSH terms highly related to COVID-19 were peripheral in the MeSH usage networks before the pandemic but have become central since 2020. We conclude that the usage patterns have changed, with COVID-19 related MeSH terms occupying a much more prominent role in 2020 than they did in the previous years.

We also contribute to the literature by estimating the effect of COVID-19 on biomedical research in a natural experiment framework, isolating the specific effects of the COVID-19 pandemic on the biomedical scientific landscape. This is crucial to identify areas of public intervention to sustain areas of biomedical research which have been neglected during the COVID-19 crisis. Moreover, the exploratory analysis on the changes in usage patterns of MeSH terms, points to an increase in the importance of covid-related topics in the broader biomedical research landscape.

Our results provide compelling evidence that research related to COVID-19 has indeed displaced scientific production in other biomedical fields of research not related to COVID-19, with a significant drop in (impact weighted) scientific output related to non-COVID-19 and a marked reduction of financial support for publications not related to COVID-19 [ 4 , 5 , 16 ]. The displacement effect is persistent to the end of 2020. As vaccination progresses, we highlight the urgent need for science policy to re-balance support for research activity that was put on pause because of the COVID-19 pandemic.

We find that COVID-19 dramatically impacted clinical research. Reactivation of clinical trials activities that have been postponed or suspended for reasons related to COVID-19 is a priority that should be considered in the national vaccination plans. Moreover, since grants have been diverted and financial incentives have been targeted to sustain COVID-19 research leading to an excessive entry in COVID-19-related clinical trials and the ‘covidisation’ of research, there is a need to reorient incentives to basic research and otherwise neglected or temporally abandoned areas of biomedical research. Without dedicated support in the recovery plans for neglected research of the COVID-19 era, there is a risk that more medical needs will be unmet in the future, possibly exacerbating the shortage of scientific research for orphan and neglected diseases, which do not belong to COVID-19-related research areas.

Limitations

Our empirical approach has some limits. First, we proxy MeSH term usage via search terms using the PubMed EUtilities API. This means that the accuracy of the MeSH term we assign to a given paper is not fully validated. More time is needed for the completion of manually annotated MeSH terms. Second, the timing of publication is not the moment the research has been carried out. There is a lead time between inception, analysis, write-up, review, revision, and final publication. This delay varies across disciplines. Nevertheless, given that the surge in publications happens around the alleged event date, January 2020, we are confident that the publication date is a reasonable yet imperfect estimate of the timing of the research. Third, several journals have publicly declared to fast-track COVID-19 research. This discrepancy in the speed of publication of COVID-19 related research and other research could affect our results. Specifically, a surge or displacement could be overestimated due to a lag in the publication of COVID-19 unrelated research. We alleviate this bias by estimating the effect considering a considerable time after the event (January 2020 to December 2020). Forth, on the one hand, clinical Trials may lead to multiple publications. Therefore we might overestimate the impact of COVID-19 on the number of clinical trials. On the other hand, COVID-19 publications on clinical trials lag behind, so the number of papers related COVID-19 trials is likely underestimated. Therefore, we note that the focus of this paper is scientific publications on clinical trials rather than on actual clinical trials. Fifth, regarding grants, unfortunately, there is no unique centralized repository mapping grant numbers to years, so we have to proxy old grants with grants that appeared in publications from 2010 to 2018. Besides, grant numbers are free-form entries, meaning that PubMed has no validation step to disambiguate or verify that the grant number has been entered correctly. This has the effect of classifying a grant as new even though it has appeared under a different name. We mitigate this problem by using a long period to collect grant numbers and catch many spellings of the same grant, thereby reducing the likelihood of miss-identifying a grant as new when it existed before. Still, unless unique identifiers are widely used, there is no way to verify this.

So far, there is no conclusive evidence on whether entry into COVID-19 has been excessive. However, there is a growing consensus that COVID-19 has displaced, at least temporally, scientific research in COVID-19 unrelated biomedical research areas. Even though it is certainly expected that more attention will be devoted to the emergency during a pandemic, the displacement of biomedical research in other fields is concerning. Future research is needed to investigate the long-run structural consequences of the COVID-19 crisis on biomedical research.

Supporting information

Full regression table with all controls and interactions.

Funding Statement

The author(s) received no specific funding for this work.

Data Availability

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Impact of COVID-19 pandemic on mental health: An international study

Roles Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

¶ ‡ ATG, MK and AK designed and implemented the study together. AK and MK should be considered joint senior authors.

Affiliation Division of Clinical Psychology & Intervention Science, Department of Psychology, University of Basel, Basel, Switzerland

Roles Data curation, Formal analysis, Writing – original draft, Writing – review & editing

Affiliation Department of Health Sciences, European University Cyprus, Nicosia, Cyprus

Roles Investigation, Resources, Writing – review & editing

Affiliation Psychological Laboratory, Faculty of Public Health and Social Welfare, Riga Stradiņš University, Riga, Latvia

Affiliation Kore University Behavioral Lab (KUBeLab), Faculty of Human and Social Sciences, Kore University of Enna, Enna, Italy

Affiliation Department of Social Sciences, School of Humanities and Social Sciences, University of Nicosia, Nicosia, Cyprus

Affiliation Department of Nursing, Cyprus University of Technology, Limassol, Cyprus

Affiliation Cyprus Institute of Neurology and Genetics, Nicosia, Cyprus

Affiliation Department of Psychological Counseling and Guidance, Faculty of Education, Hasan Kalyoncu University, Gaziantep, Turkey

Affiliation The Nethersole School of Nursing, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong

Affiliation Department of Psychology, Fundación Universitaria Konrad Lorenz, Bogotà, Columbia

Roles Conceptualization, Investigation, Resources, Writing – review & editing

Affiliation Faculty of Psychology, University of La Sabana, Chía, Columbia

Affiliation School of Applied Psychology, University College Cork, Cork, Ireland

Affiliation School of Psychology, University College Dublin, Dublin, Ireland

Affiliation Medical University Innsbruck, Innsbruck, Austria

Affiliation Department of Psychology, Babeş-Bolyai University (UBB), Cluj-Napoca, Romania

Affiliation Instituto Superior de Psicologia Aplicada (ISPA), Instituto Universitário; APPsyCI—Applied Psychology Research Center Capabilities & Inclusion, Lisboa, Portugal

Affiliation Faculdade de Psicologia, Alameda da Universidade, Universidade de Lisboa, Lisboa, Portugal

Affiliation LIP/PC2S, Université Grenoble Alpes, Grenoble, France

Affiliation Department of Biomedicine, Biotechnology and Public Health, University of Cadiz, Cadiz, Spain

Affiliation Instituto ACT, Madrid, Spain

Affiliation Department of Psychology, European University of Madrid, Madrid, Spain

Affiliation Department of Psychology and Sociology, University of Zaragoza, Zaragoza, Spain

Affiliation Vadaskert Child and Adolescent Psychiatric Hospital, Budapest, Hungary

Affiliation Private Pratice, Poland

Affiliation Department of Psychology, University of Jyväskylä, Jyväskylä, Finland

Affiliation Clinic for Psychiatry, Clinical Center of Montenegro, Podgorica, Montenegro

Affiliation Ljubljana University Medical Centre, Ljubljana, Slovania

Affiliation Département de Psychologie, Université du Québec à Trois-Rivières, Trois-Rivières, Canada

Affiliation Department of Psychiatry and Behavioral Science, Duke University, Durham, North Carolina, United States of America

Roles Conceptualization, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

Affiliation Department of Psychology, University of Cyprus, Nicosia, Cyprus

•  [ ... ],
• [ view all ]
• [ view less ]
• Andrew T. Gloster, 
• Demetris Lamnisos, 
• Jelena Lubenko, 
• Giovambattista Presti, 
• Valeria Squatrito, 
• Marios Constantinou, 
• Christiana Nicolaou, 
• Savvas Papacostas, 
• Gökçen Aydın, 

• Published: December 31, 2020
• https://doi.org/10.1371/journal.pone.0244809
• Reader Comments

The COVID-19 pandemic triggered vast governmental lockdowns. The impact of these lockdowns on mental health is inadequately understood. On the one hand such drastic changes in daily routines could be detrimental to mental health. On the other hand, it might not be experienced negatively, especially because the entire population was affected.

The aim of this study was to determine mental health outcomes during pandemic induced lockdowns and to examine known predictors of mental health outcomes. We therefore surveyed n = 9,565 people from 78 countries and 18 languages. Outcomes assessed were stress, depression, affect, and wellbeing. Predictors included country, sociodemographic factors, lockdown characteristics, social factors, and psychological factors.

Results indicated that on average about 10% of the sample was languishing from low levels of mental health and about 50% had only moderate mental health. Importantly, three consistent predictors of mental health emerged: social support, education level, and psychologically flexible (vs. rigid) responding. Poorer outcomes were most strongly predicted by a worsening of finances and not having access to basic supplies.

Conclusions

These results suggest that on whole, respondents were moderately mentally healthy at the time of a population-wide lockdown. The highest level of mental health difficulties were found in approximately 10% of the population. Findings suggest that public health initiatives should target people without social support and those whose finances worsen as a result of the lockdown. Interventions that promote psychological flexibility may mitigate the impact of the pandemic.

Citation: Gloster AT, Lamnisos D, Lubenko J, Presti G, Squatrito V, Constantinou M, et al. (2020) Impact of COVID-19 pandemic on mental health: An international study. PLoS ONE 15(12): e0244809. https://doi.org/10.1371/journal.pone.0244809

Editor: Joel Msafiri Francis, University of the Witwatersrand, SOUTH AFRICA

Received: October 3, 2020; Accepted: December 16, 2020; Published: December 31, 2020

Copyright: © 2020 Gloster et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This work was supported by grants from the Swiss National Science Foundation awarded to Andrew T. Gloster (PP00P1_ 163716/1 & PP00P1_190082). The funder provided support in the form of salaries for authors [ATG], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section. One of the authors is employed by a commercial affiliation: Private Pratice, Poland. This affiliation provided support in the form of salaries for authors [BK], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: One of the authors is employed by a commercial affiliation: Private Pratice, Poland. This affiliation provided support in the form of salaries for author BK, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. This does not alter our adherence to PLOS ONE policies on sharing data and materials. No other authors have competing interests to declare.

Introduction

The COVID-19 global pandemic caused by the severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) virus triggered governmentally mandated lockdowns, social distancing, quarantines and other measures in the interest of public health. The mandated lockdowns abruptly and dramatically altered people’s daily routines, work, travel, and leisure activities to a degree unexperienced by most people living outside of war zones. Simultaneously, the highly contagious, yet invisible virus transformed previously neutral situations to perceived potentially dangerous ones: social interaction, touching one’s face, going to a concert, shaking someone’s hand, and even hugging grandparents. Given these changes and looming threat, increases in anxiety and depression can be expected [ 1 ]. Indeed, common psychological reactions to previous quarantines include post-traumatic symptoms, confusion, and anger [ 2 ], though these data stem from quarantines of specific regions or a subgroup of exposed people, such as medical professionals. It therefore remains an empirical question whether such patterns are consistent when entire populations across the globe are simultaneously affected.

For most people, it stands to reason that governmentally mandated lockdowns decrease their activity levels and the number of stimuli experienced compared to pre-lockdown levels. The impact of reducing activities, stimuli and routines on the population is unknown, but various analogue situations can be used to make predictions, like death of a spouse [ 3 ]; hearing loss [ 4 ]; job loss [ 5 ]; long duration expeditions [ 6 ]; poor acculturation [ 7 ]; and even ageing when combined with loneliness [ 8 ]. Each of these situations is associated with increases in psychological distress. This reduction of stimulations may lead to boredom and reductions in reinforcement, which has been associated with depression [ 9 ]. The sum total of these literatures, and some evidence from country specific studies on COVID-19 suggests that for some people, the mental distress in the form of stress, depression, and negative affect are likely reactions to the lockdown; therefore, people’s wellbeing is likely to suffer. Indeed, increased loneliness, social isolation, and living alone are associated with increased mortality [ 10 ]–the exact effect that mandated lockdown and social distancing rules aimed to counteract.

Alternately, the planned slowing down of daily routines can be beneficial. For example, vacations and weekends are highly sought-after–if not always achieved–periods of relaxation and stress reduction [ 11 ]. Likewise, some religious and spiritual traditions encourage simplicity, mindfulness, and solitude with the goal of increasing wellbeing [ 12 ]. It is therefore conceivable that for some people the lockdown could offer a reprieve from daily hassles and stress and even lead to increases in wellbeing. It is therefore equally important to identify protective factors that can buffer against the negative effects of the lockdown.

Although nearly all people around the globe have been subject to some form of lockdown measures to contain the COVID-19 response, variations exist with respect to how each person is confined, even within a single country. For instance, during the COVID-19 pandemic some people were allowed to go to work, whereas others were required to work exclusively from home. For various reasons, some people had difficulty obtaining some basic supplies. Further, some were thrust into the situation of taking care of others (e.g., children, due to closing of schools). Finally, some people lost income as a result of the lockdown, and this is a known risk-factor for poor mental health [ 13 , 14 ]. Finally, a lockdown may be experienced differently the longer it continues and potentially when in confined spaces [ 2 ]. All of these lockdown-specific features may have an impact on one’s mental health, but to date it remains inadequately explored.

As the risk of the pandemic continues, it is important to understand to what degree the virus-induced uncertainty and the lockdown-induced changes in daily routines impact stress, depression, affect, and wellbeing. Towards this end, it is important to identify factors that can mitigate potential negative psychological effects of pandemics and lockdowns. Various social and psychological factors have been identified in other contexts that may also help build resilience in large-scale pandemics such as COVID-19. On the social level, one such candidate is social support, which has repeatedly been found to positively impact mental health and wellbeing [ 15 – 18 ]. Another social factor is the family climate and family functioning, which clearly impacts people’s mental health [ 19 , 20 ]. Psychological factors such as mindfulness and psychologically flexible response styles (as opposed to rigid and avoidant response styles) are behavioral repertoires that have previously been shown to buffer the impact of stress and facilitate wellbeing [ 21 – 24 ].

Given the scope of the COVID-19 pandemic, it is crucial to better understand how a pandemic and associated lockdowns impact on mental health. Thus, the aim of this study was to determine mental health outcomes and to examine known predictors of outcomes to identify psychological processes and contextual factors that can be used in developing public health interventions. It can be assumed, but remains untested, that those with risks in social-demographic factors, living conditions, social factors and psychological factors have more severe reactions to the lockdown. We therefore tested whether outcomes of stress, depression, affect, and wellbeing were predicted by country of residence, social demographic characteristics, COVID-19 lockdown related predictors, social predictors, and psychological predictors.

Participants

The inclusion criteria were ≥18 years of age and ability to read one of the 18 languages (English, Greek, German, French, Spanish, Turkish, Dutch, Latvian, Italian, Portuguese, Finnish, Slovenian, Polish, Romanian, Hong Kong, Hungarian, Montenegrin, & Persian.). There were no exclusion criteria. People from all countries were eligible to participate.

Ethics approval was obtained from the Cyprus National Bioethics Committee (ref.: EEBK EΠ 2020.01.60) followed by site approvals from different research teams involved in data collection. All participants provided written informed consent prior to completing the survey (computer-based, e.g., by clicking “yes”).

A population based cross-sectional study was conducted in order to explore how people across the world reacted to the COVID-19. The anonymous online survey was distributed using a range of methods. Universities emailed the online survey to students and academic staff and also posted the survey link to their websites. In addition, and in order to broaden the sample to older age groups and to those with different socio-demographic characteristics, the survey was disseminated in local press (e.g., newspapers, newsletters, radio stations), in social media (e.g., Facebook, Twitter, etc.), in professional networks, local hospitals and health centers and professional groups’ email lists (e.g., medical doctors, teachers, engineers, psychologists, government workers), and to social institutions in the countries (e.g., churches, schools, cities/townships, clubs, etc.).

Data were collected for two months between 07th April and 07th June 2020. The majority of countries where data were collected had declared a state of emergency for COVID-19 during this time.

Well validated and established measures were used to assess constructs. When measures did not already exist in a language, they were subject to forward and backward translation procedures. Well-validated measures of predictors and outcomes and items measuring COVID-19 related characteristics were selected after a consensus agreement among the members of this study.

Respondents’ countries were coded and entered as predictors.

Socio-demographic status.

Participants responded to questions related to their socio-demographic characteristics including their age, gender, country of residence, marital status, employment status, educational level, whether they have children as well as their living situation.

Lockdown variables.

Participants responded to questions related to lockdown including length of lockdown, whether they need to leave home for work, any change in their finances, whether they were able to obtain basic supplies, the amount of their living space confined in during the lockdown. They were also asked whether they, their partner, or a significant other was diagnosed with COVID-19.

Social factors.

Social factors were measured using the Brief Assessment of Family Functioning Scale (BAFFS; [ 25 ]) and the Oslo Social Support Scale (OSSS; [ 26 ]). The BAFFS items are summed to produce a single score with higher scores indicating worse family functioning. The OSSS items are summed up and provide three levels types of social support: low (scored 3–8), moderate (scored 9–11) and high (scored 12–14).

Psychological factors.

Psychological factors including mindfulness and psychological flexibility. Mindfulness was measured using the Cognitive Affective Mindfulness Scale (CAMS; [ 27 ]). The CAMS produces a single score with higher scores indicating better mindfulness qualities. Psychological flexibility (e.g., hold one’s thoughts lightly, be accepting of one’s experiences, engage in what is important to them despite challenging situations) was measured using the Psyflex scale [ 28 ]. The Psyflex produces a single score with higher scores indicating better psychological flexibility qualities.

Stress was measured using the Perceived Stress Scale (PSS; [ 29 ]). The PSS assesses an individual’s appraisal of how stressful situations in their life are. Items ask about people’s feelings and thoughts during the last month. A total score is produced, with higher scores indicating greater overall distress.

Depression.

Depressive symptomatology was assessed using two items from the disengagement subscale of the Multidimensional State Boredom Scale (MSBS; [ 30 ]). These items assessed wanting to do pleasurable things but not finding anything appealing (i.e., boredom), as well as wasting time. Based on concepts of reinforcement deprivation (i.e., lack of access to or engagement with positive stimuli) that is known to contribute to depression, we added an item that measured how rewarding or pleasurable people found the activities that they were engaging in (i.e., reinforcement). Higher scores indicated higher depressive symptomatology.

Positive affect/ negative affect.

The Positive And Negative Affect Scale (PANAS) was used to measure affect [ 31 ]. The original version of the questionnaire was used with five additional items: bored, confused, angry, frustrated and lonely. All items were scored on a 5-point Likert type scale, ranging from 1 = very little/not at all to 5 = extremely and summed up so that higher scores in the positive-related items indicating higher positive affect and higher scores in the negative-related items indicating higher negative affect. In order to capture additional dimensions of negative affect believed to be relevant to the COVID-19 lockdowns, we additionally added five items: bored, confused, angry, frustrated, lonely.

Wellbeing was assessed using the Mental Health Continuum Short Form (MHC-SF; [ 32 ]); which assesses three aspects of wellbeing: emotional, psychological, and social. The MHC-SF produces a total score and scores for each of the three aspects of wellbeing. The MHC-SF can also be scored to produce categories of languishing (i.e., low levels of emotional, psychological, and social well-being), flourishing (i.e., high levels of emotional psychological and social well-being almost every day), and moderately mentally healthy (in between languishing and flourishing).

Statistical analysis

The mean and standard deviation was calculated for dependent variables that follow the normal distribution while the median and interquartile range (IQR) were computed for non-normally distributed data. Bivariable association between an outcome variable and each predictor was investigated with ANOVA test for categorical predictor and univariable linear regression for numerical predictor. Linear mixed-effect model with random effect for country was performed to consider simultaneously several predictors in the same model and to account for the variation in outcome variable between countries. Four separate linear mixed-effect models were used for each outcome variable, one for each set of socio-demographic, lockdown, social and psychosocial predictors and multicollinearity for each set of predictors was investigated with the variation inflation criterion (VIF). Standardized regression coefficients were computed as effect size indices to measure the strength of the association between predictor variables and outcome variables. The comparison between the country mean and overall mean for each outcome variable was estimated though a linear regression model with dependent variable the mean centering outcome and predictor the country. Cohen’s d effect size of the standardize difference between country mean and the overall mean was computed as a measure of the magnitude of the difference between the two means.

The whole sample was used in linear mixed-effect models while for the comparison of country mean to the overall mean was used the sample from countries with sample size ≥100. The R packages lme4 and effect sizes were used for fitting the linear mixed effect model and to compute the standardized regression coefficients of the linear mixed effect models [ 33 ]. Significance test and confidence intervals were calculated at a significance level of 0.05. The following cut-off values were used for the evaluation of the effect sizes: ‘tiny’ ≤0.05, ‘very small’ from 0.05 to ≤0.10, ‘small’ from 0.10 to ≤ 0.20, ‘medium’ from 0.20 to ≤ 0.30, ‘large’ from 0.30 to ≤ 0.40 and ‘very large’ > 0.40 [ 34 ].

Descriptive

Participants were n = 9,565 people from 78 countries. See supporting information for a participation flowchart ( S1 Appendix ). The countries with the largest samples were: Latvia (n = 1285), Italy (n = 962), Cyprus (n = 957), Turkey (n = 702), Switzerland (n = 550), Hong Kong (n = 516), Colombia (n = 485), Ireland (n = 414), Austria (n = 368), Romania (n = 339), Portugal (n = 334), France (n = 313), Spain (n = 296), Germany (n = 279), Hungary (n = 273), Greece (n = 270), USA (n = 268), Finland (n = 157), Montenegro (n = 147), Poland (n = 135), United Kingdom (n = 100), Slovenia (n = 77), and Canada (n = 60). The remaining countries are listed in the supporting information ( S1 Table ).

Outcome variables

The means, standard deviations, and where appropriate percentage of participants within categories of the five outcome variables can be seen in Table 1 .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0244809.t001

Predictor variables

A full list of countries can be found in the supporting information ( S1 Table ).

The mean age was 36.9 (13.3) years. A majority of participants were female (77.7%), approximately a fifth male (22.0%), and small minority identified as other (0.3%). More than half of the respondents were either in a relationship (25.7%) or married (36.1%), almost a third were single (30.8%), and the rest were either divorced (5%), widower (1.1%) or other (1.3%). Participants indicated that they lived: alone (14.6%), with both parents (20.8%), one parent (5.1%), with their own family including partner and children (54.1%), or with friends or roommates (5.5%). Less than half of respondents had children (40.8%). Approximately half of the participants were working full time (53.4%), almost a fifth were working part-time (17.5%), 23.2% were unemployed and a small minority were either on parental leave (2.2%) or retired (3.7%).

COVID-19 lockdown variables.

At the time of responding, participants were in lockdown or self-isolation for a median of 5.0 (3.0 IQR) weeks. Most people indicated that they had not been infected with COVID-19 (88.0%), a small minority indicated they had been infected (1.4%) and the rest had symptoms but were unsure (10.6%). Similar patterns were seen with reported infection rates of partners (no: 92.2%, yes: 0.7%, unsure: 7.1%) and of people close to them (no: 86.0%; yes: 5.6%; unsure: 8.4%). With respect to leaving the house for work, almost half (47.7%) indicated that this never occurred, 7.7% indicated leaving only once, whereas an almost equal number indicated leaving a couple times per week (23.7%) or more than three times per week (21.0%). Nearly all participants indicated they were able to obtain all the basic supplies they needed (93.5%). Participants reported having a median inner living space of 90.0 square meters (80.0 IQR) and median outdoor space of 20.0 square meters (192.1 IQR). Finally, with respect to finances, more than half indicated that their financial situation remained about the same (57.9%), a minority indicated it improved (8.9%), and a third reported that their finances had gotten worse (33.3%).

Social and psychological predictors.

Mean values of the other predictors (i.e., social predictors and psychological predictors) can be seen in Table 1 .

Multivariate analyses

Results of multivariate analyses for the outcome of stress can be seen in Table 2 . The largest protective factor against stress was social support (high support vs low support (-3.35, 95%CI, -3.39 to -2.92), with a very large effect size). Positive predictors of stress with large effect sizes were being female (2.42, 95%CI, 2.07 to 2.77) and worsening of finances (2.32, 95%CI, 1.68 to 2.96), whereas psychological flexibility buffered this response (-0.65, 95%CI, -0.69 to -0.62). Higher education levels were also associated with lower levels of stress, with a large effect size (see Table 2 ). Moderate effect sizes for predictors associated with less stress were older age (-0.13, 95%CI, -0.14, -0.11) and mindfulness (-0.69, 95%CI, -0.74, -0.64). Moderate effect sizes of predictors associated with more stress were worse family functioning (0.98, 95%CI, 0.90, 1.06) and not being able to obtain all basic supplies (1.82 95%CI, 1.12, 2.52).

https://doi.org/10.1371/journal.pone.0244809.t002

Differences in reported levels of stress across countries were largely negligible, with the exception of two countries that reported higher levels of stress (Hong Kong (2.85, 95%CI, 2.22, 3.49) and Turkey (2.47, 95%CI, 1.93, 3.02)) and two that reported lower levels of stress (Portugal (-2.50, 95%CI, -3.29, -1.71) and Montenegro (-3.30, 95%CI, -4.49, -2.11)) than the average stress level across all countries. See supporting information for information on each country ( S2 – S6 Tables).

Results of multivariate analyses for the outcome of depression can be seen in Table 3 . The strongest predictor of depression was social support, such that high (-1.30, 95%CI, -1.44, -1.16) and medium levels (-0.73, 95%CI, -0.85, -0.62) of social support were protective against depression (relative to low levels) with a very large and large effect sizes, respectively. The only other large effect size was for psychological flexibility, which also served in a protective manner (-0.20, 95%CI, -0.22, -0.19). Moderate effect sizes of predictors associated with less depression symptoms were also observed for higher education levels (see Table 3 ). Moderate effect sizes of predictors associated with more depression were worse family functioning (0.29, 95%CI, 0.27, 0.32) and not being able to obtain all basic supplies (0.49, 95%CI, 0.27, 0.70).

https://doi.org/10.1371/journal.pone.0244809.t003

The amount of depression symptoms reported on average within countries was similar for most countries with the exception of one country with lower reported levels than average with a large effect size (Austria (-0.71, 95%CI, -0.95, -0.47)) and one with higher levels than average with a large effect size (USA (0.85, 95%CI, 0.58, 1.13)). See supporting information for information on each country ( S2 – S6 Tables).

Results of multivariate analyses for the outcome of affect can be seen in Table 4 . With respect to positive affect, social support (high support vs low support (5.69, 95%CI, 5.23, 6.16) and psychological flexibility (0.77, 95%CI, 0.74, 0.81) were both predictors with very large effect sizes. Interestingly, those who left their house more than three times per week had higher levels of positive affect than those that did not leave their house for work (1.68, 95%CI, 1.18, 2.17), with a medium effect size. Higher education levels were associated with higher levels of positive affect with a medium to large effect size (see Table 4 , PANAS-Positive).

https://doi.org/10.1371/journal.pone.0244809.t004

The amount of positive affect reported on average within countries was similar for most countries with the exception of one country with lower reported levels than average with a large effect size (Finland (-2.96, 95%CI, -4.19, -1.73)) and one with higher reported levels than average with a large effect size (Portugal (2.96, 95%CI, 2.12, 3.80)). See supporting information for information on each country ( S2 – S6 Tables).

With respect to negative affect, social support (high support vs low support (-2.74, 95%CI, -3.2, -2.29) and psychological flexibility (-0.62, 95%CI, -0.66, -0.58) were again the strongest associated predictors, with large effects. Higher education levels were also associated with lower levels of negative affect, with a medium effect (see Table 4 , PANAS-Negative). Higher levels of negative affect were noted, with medium effect sizes, for the predictors: worsening of finances (1.75, 95%CI, 1.10, 2.40) and not being able to obtain all basic supplies (1.6, 95%CI, 0.89, 2.31).

The amount of negative affect reported on average within countries was similar for most countries with the exception of few countries with lower reported negative affect levels than average with a very large effect sizes (Switzerland (-4.96, 95%CI, -5.91, -4.01), Germany (-4.70, 95%CI, -6.03, -3.37) & Austria (-6.49, 95%CI, -7.65, -5.33)) and one with a large effect size (Montenegro (-3.56, 95%CI, -5.39, -1.73). The average amount of negative affect was higher than average in two countries, with very large effects size (Turkey (5.75, 95%CI, 4.92, 6.59) & Finland (7.57, 95%CI, 5.80, 9.34)). See supporting information for information on each country ( S2 – S6 Tables).

Results of multivariate analyses for the outcome of wellbeing can be seen in Table 5 . Once again, social support (high support vs low support (13.20, 95%CI, 12.39, 14.01)) and psychological flexibility (1.42, 95%CI, 1.34, 1.49) were the predictors with the largest effect sizes (very large) on wellbeing. Higher education levels were associated with higher levels of wellbeing with a medium to large effect sizes (see Table 5 ). Medium negative effect sizes were noted for family functioning (-1.98, 95%CI, -2.12, -1.83) and inability to obtain all basic supplies (-3.27, 95%CI, -4.67, -1.87). Two medium positive effect sizes were observed: mindfulness (0.95, 95%CI, 0.86–1.04) and living with friends/roommates ((3.04, 95%CI, 1.59, 4.48), relative to living alone).

https://doi.org/10.1371/journal.pone.0244809.t005

The level of wellbeing reported on average within countries was similar for most countries with the exception of three countries with higher levels with large effect sizes (Austria (4.95, 95%CI, 3.55, 6.34), Finland (5.24, 95%CI, 3.10, 7.38), & Portugal (4.59, 95%CI, 3.12, 6.05)) and two countries with lower levels of wellbeing than average with large (Italy (-4.36, 95%CI, -11.06, 2.35)) and very large effect sizes (Hong Kong (-6.84, 95%CI, -8.02, -5.66)). See supporting information for information on each country ( S2 – S6 Tables).

The COVID-19 is the largest pandemic in modern history. This study assessed nearly 10,000 participants across many countries to examine the impact of the pandemic and resultant governmental lockdown measures on mental health. During the height of the lockdown, the pandemic was experienced as at least moderately stressful for most people, and 11% reported the highest levels of stress. Symptoms of depression were also high, including 25% of the sample indicating that the things they did were not reinforcing, 33% reporting high levels of boredom, and nearly 50% indicating they wasted a lot of time. Consistent with symptoms of stress and depression, 10% of participants were psychologically languishing. These results suggest that there is a subgroup of people who are especially suffering and that in about 50% of the respondents’ levels of mental health was only moderate. Previous studies have found that along with low levels, even moderate levels of mental health (which consists of only moderate levels of emotional, psychological, and social well-being) are associated with increased subsequent disability, productivity loss, and healthcare use [ 35 – 37 ]. Not everyone was suffering, however, as evidenced by the nearly 40% of participants who reported levels of mental health consistent with flourishing. The present results, while serious, do not point to more severe reactions observed in previous samples of selective quarantined individuals or groups [ 2 ]. Perhaps the previously reported distress in these groups is prevented when an entire country or world is in lockdown so that the feeling emerges that “everyone is in it together”.

Importantly, a handful of predictors emerged that consistently predicted all outcomes: Social support, education level, finances, access to basic needs, and the ability to respond psychologically flexible. The consistency of results examining predictors is noteworthy, both in terms of the consistently strong predictors (e.g., social support, education, psychological flexibly, as well as loss of income and lack of access to necessities) and in terms of the other predictors that were either not predictive or only weakly so. All predictors were chosen based on theoretical ties to the outcomes, previous findings, and studies on quarantines [ 2 ].

A novel finding was that people who left their house three or more times per week reported more positive affect than those that left their house less often. It is possible that these people experienced more variation, which contributed to positive affect. It is also possible they experienced a greater sense of normality. Future studies are encouraged to further investigate possible mechanisms through which this result unfolds.

Overall, these patterns did not differ substantially between countries. Although some differences did emerge, they were mostly inconsistent across outcomes. Three countries fared worse on two outcomes each: Hong Kong (stress & wellbeing); Turkey (stress & negative affect); and Finland (lower positive affect and higher negative affect)–though participants in Finland also reported higher levels of wellbeing than average. Two countries had more favorable outcomes than the average levels across all countries: Portugal (lower stress and higher wellbeing) and Austria (lower depression and higher wellbeing). The differences observed are likely due to a combination of chance, sampling, nation specific responses to the COVID-19 pandemic, cultural differences, and other factors playing out in the countries (e.g., political unrest [ 38 ]). If replicated, future studies are encouraged to examine possible mechanisms of these outcomes.

This study provides valuable insights on several levels. First, it documents the mental health outcomes across a broad sample during the COVID-19 global pandemic. Second, it informs about the conditions and resilience factors (social support, education, and psychological flexibility) and risk factors (loss of income and inability to get basic supplies) that affect mental health outcomes. Third, these factors can be used in future public health responses are being made, including those that require large scale lockdowns or quarantines. That is, public health officials should direct resources to identifying and supporting people with poor social support, income loss, and potentially lower levels of education and provide a strategy to mitigate special risks in these subpopulations. The importance of social support needs to be made clear to the public and to the degree possible mechanisms that can contribute to social support should be supported. Further, psychological flexibility is a trainable set of skills that has repeatedly been shown to ameliorate suffering [ 22 , 39 ]; and can be widely distributed with modern technological intervention tools such as digital, internet, or virtual means [ 40 ]. We do not claim, however, that psychological flexibility is the only factor that can be used for interventions. Instead, it is a recognized transdiagnostic factor assessed in this study and one that is feasible to be targeted and modified by interventions and prevention [ 41 – 43 ].

This study is limited by several important factors. First, the results are based on cross sectional analysis and correlations. As such, causation cannot be inferred and any delayed impact of the pandemic and lockdown on peoples’ mental health was not captured. Second, all results of this survey were obtained via self-report questionnaires, which can be subject to retrospective response bias. Third, although the sample was large and based on varied recruitment sources, it was not representative of the population and undersampled people who suffered most from the pandemic (i.e., front line health care professionals, people in intensive care, etc.) or people without internet access, etc. Finally, the country-specific incidence rates and lockdown measures differed across countries. These were not assessed, but future studies are encouraged to investigate how such factors impact mental health outcomes.

These limitations notwithstanding, based on nearly 10,000 international participants, this study found that approximately 10% of the population was languishing during or shortly after the lockdown period. These finding have implications for public health initiatives. First, officials are urged to attend to, find, and target people who have little social support and/ or whose finances have worsened as a result of the measures. Second, public health interventions are further urged to target psychological processes such as psychological flexibility in general to potentially help buffer other risk factors for mental health. Likewise, availability of social support and information about where to get support and remain connected are needed. These recommendations should become part of public health initiatives designed to promote mental health in general, and should equally be considered when lockdowns or physical distancing are prescribed during a pandemic.

Supporting information

S1 table. list of all countries included in the data set..

https://doi.org/10.1371/journal.pone.0244809.s001

S2 Table. Geodemographic predictors for Perceived Stress Scale.

https://doi.org/10.1371/journal.pone.0244809.s002

S3 Table. Geodemographic predictors for MSBS–depression.

https://doi.org/10.1371/journal.pone.0244809.s003

S4 Table. Geodemographic predictors for PANAS positive.

https://doi.org/10.1371/journal.pone.0244809.s004

S5 Table. Geodemographic predictors for PANAS negative.

https://doi.org/10.1371/journal.pone.0244809.s005

S6 Table. Geodemographic predictors for MHCSF—mental health continuum.

https://doi.org/10.1371/journal.pone.0244809.s006

S1 Appendix. Participation flowchart.

https://doi.org/10.1371/journal.pone.0244809.s007

Acknowledgments

We wish to thank the following people for their work in helping to implement the study: Spyros Demosthenous, Christiana Karashali, Diamanto Rovania (University of Cyprus); Maria Antoniade (European University of Cyprus); Ioanna Menoikou (Cyprus University of Technology); Elias Ioannou (University of Nicosia); Sonja Borner, Victoria Firsching-Block, Alexander Fenn (University of Basel); Cristīne Šneidere, Ingrīda Trups-Kalne, Lolita Vansovica, Sandra Feldmane, (Riga Stradiņš University); David Nilsson (Lund University); Miguel A. Segura-Vargas (Fundación Universitaria Konrad Lorenz); Claudia Lenuţa Rus, Catalina Otoiu, Cristina Vajaean (Babes-Bolyai University). We further wish to thank Fabio Coviello and Sonja Borner (University of Basel) for their help in preparing the manuscript.

• View Article
• Google Scholar
• 2. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Vol. 395, The Lancet. Lancet Publishing Group; 2020. p. 912–20.
• 3. Kristensen P, Weisaeth L, Heir T. Bereavement and mental health after sudden and violent losses: A Review. Vol. 75, Psychiatry. Guilford Publications Inc; 2012. p. 76–97. https://doi.org/10.1521/psyc.2012.75.1.76 pmid:22397543
• PubMed/NCBI
• 6. Alfano CA, Bower JL, Cowie J, Lau S, Simpson RJ. Long-duration space exploration and emotional health: Recommendations for conceptualizing and evaluating risk. Vol. 142, Acta Astronautica. Elsevier Ltd; 2018. p. 289–99.
• 9. Kanter JW, Busch AM, Weeks CE, Landes SJ. The nature of clinical depression: Symptoms, syndromes, and behavior analysis. Vol. 31, Behavior Analyst. Association for Behavior Analysis; 2008. p. 1–21.
• 12. Koenig HG. Research on religion, spirituality, and mental health: A review. Vol. 54, Canadian Journal of Psychiatry. Canadian Psychiatric Association; 2009. p. 283–91. https://doi.org/10.1177/070674370905400502 pmid:19497160
• 23. Gu J, Strauss C, Bond R, Cavanagh K. How do mindfulness-based cognitive therapy and mindfulness-based stress reduction improve mental health and wellbeing? A systematic review and meta-analysis of mediation studies. Vol. 37, Clinical Psychology Review. Elsevier Inc.; 2015. p. 1–12. https://doi.org/10.1016/j.cpr.2015.01.006 pmid:25689576
• 29. Cohen S. Perceived stress in a probability sample of the United States. In: The social psychology of health. Thousand Oaks, CA, US: Sage Publications, Inc; 1988. p. 31–67. (The Claremont Symposium on Applied Social Psychology.).
• 43. Dindo L, Van Liew JR, Arch JJ. Acceptance and Commitment Therapy: A Transdiagnostic Behavioral Intervention for Mental Health and Medical Conditions. Vol. 14, Neurotherapeutics. Springer New York LLC; 2017. p. 546–53. https://doi.org/10.1007/s13311-017-0521-3 pmid:28271287

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 February 2022

Effect of COVID-19 on agricultural production and food security: A scientometric analysis

• Collins C. Okolie   ORCID: orcid.org/0000-0002-6633-6717 1 &
• Abiodun A. Ogundeji   ORCID: orcid.org/0000-0001-7356-5668 1  

Humanities and Social Sciences Communications volume  9 , Article number:  64 ( 2022 ) Cite this article

29k Accesses

39 Citations

1 Altmetric

Metrics details

• Environmental studies

Coronavirus disease has created an unexpected negative situation globally, impacting the agricultural sector, economy, human health, and food security. This study examined research on COVID-19 in relation to agricultural production and food security. Research articles published in Web of Science and Scopus were sourced, considering critical situations and circumstance posed by COVID-19 pandemic with regards to the shortage of agricultural production activities and threat to food security systems. In total, 174 published papers in BibTeX format were downloaded for further study. To assess the relevant documents, authors used “effects of COVID-19 on agricultural production and food security (ECAP-FS) as a search keyword for research published between 2016 and April 2021 utilising bibliometric innovative methods. The findings indicated an annual growth rate of about 56.64%, indicating that research on ECAP-FS increased over time within the study period. Nevertheless, the research output on ECAP-FS varied with 2020 accounting for 38.5%, followed by 2021 with 37.9% as at April 2021. The proposed four stage processes for merging two databases for bibliometric analyses clearly showed that one can run collaboration network analyses, authors coupling among other analyses by following our procedure and finally using net2VOSviewer, which is embedded in Rstudio software package. The study concluded that interruptions in agricultural food supply as a result of the pandemic impacted supply and demand shocks with negative impacts on all the four pillars of food security.

Similar content being viewed by others

Exploring the evolving landscape of COVID-19 interfaced with livelihoods

Perspectives of scholars on the origin, spread and consequences of COVID-19 are diverse but not polarized

Big data visualisation in regional comprehensive economic partnership: a systematic review

Introduction.

The coronavirus disease (COVID-19) has created an unusual situation globally (Alam and Khatun, 2021 ). Barely a year ago early in the year 2020, the unusual nature of coronavirus caused most governments to implement stringent steps in their countries to restrain the virus’s spread. The novel coronavirus (SARS-CoV-2) disease impacted economies throughout the world, disproportionately impacting individuals who were already susceptible to poverty and hunger (Laborde et al., 2020a ; Ceballos et al., 2020 ). In late December 2019, the virus was discovered in Wuhan City, Hubei Province, China. The pandemic caused by COVID-19 presented a major danger to human health, the economy, and food security in both industrialised and emerging nations (Mottaleb et al., 2020 ; Carroll et al., 2020 ; Alam and Khatun, 2021 ). Lessons learned from China revealed that various COVID-19 countermeasures such as lockdown in the country hampered production. This poses a significant risk to the long-term food supply (FAO, 2020 ), and has a negative impact on the economy, resulting in economic decline and crisis (Bai, 2020 ). It is important to understand that certain precautional and control efforts compromise agricultural production (Singh et al., 2021 ).

The virus wreaked havoc on the agricultural production sector, which is at the heart of the food chain (Pu and Zhong, 2020 ). The global spread of coronavirus resulted in the greatest economic downturn since World War Two (Hanna et al., 2020 ; Xu et al., 2021 ). The epidemic’s major impact on agricultural labour input was the restriction of labour mobility. Farmers were not permitted to just go out and gather in any way except to purchase essentials. This resulted in a manpower scarcity and reduced mass production efficiency. For instance, due to a scarcity of migrant experts, producers from Sichuan, Hunan, and Hubei in the grain-producing districts in China (south-eastern coastal district) were not able to sow their crops in good time (Pu and Zhong, 2020 ). Furthermore, wheat and pulse harvesting in northwest India was hampered due to a lack of migrant labour (Dev, 2020 ). Vegetable farmers in Ethiopia incurred not just financial loss as a result of overstocked items, but also from a lack of vital inputs (Tamru et al., 2020 ). Before the pandemic, suppliers may have planted six hectares in a single day, but due to the difficulties in finding tractor drivers during the pandemic, they were only able to cover three hectares a day (Pu and Zhong, 2020 ). Any interruptions in agricultural food supply will indeed result in supply and demand shocks, which will have an immediate effect on the agricultural sector of the economy with long-term economic performance and food security implications (Gregorio and Ancog, 2020 ).

Food security refers to a situation where all individuals at all time have continuous physical and economic access to sufficient, safe, and nutritious food to fulfil their dietary needs and food choices for an active and healthy lifestyle (Elsahoryi et al., 2020 ). Food security has been jeopardised both directly and indirectly as a result of the virus’s destabilisation of food systems and the effects of lockdowns on family revenue and physical access to food (Devereux et al., 2020 ). The presence of coronavirus disease has a negative impact on all the four pillars of food security, viz. availability of food, accessibility of food, utilisation of food, and stability of food (Nechifor et al., 2021 ; Laborde et al., 2020b ). According to Genkin and Mikheev ( 2020 ), the report by the Food and Agriculture Organization (FAO), World Trade Organization, and World Health Organization (WHO) note the threat of a food catastrophe triggered by the current coronavirus pandemic, with a risk of a global “food shortage” owing to interruptions in the trade industry’s supply chain. According to the report, global commerce contracted by roughly 20% in 2020, with 90–120 million human beings falling into severe destitution and over 300 million facing food security issues in emerging nations. To combat the COVID-19 pandemic, world leaders implemented steps to decrease the number of commodities carried by sea, air and land, as well as labour migration at national and global levels. These variables contributed to a widespread disturbance in agricultural output and food distribution systems, posing challenges to the transportation of food and agricultural resources (Genkin and Mikheev, 2020 ).

Present literature centred on the effect of coronavirus on food security or effect of coronavirus on agricultural production (Elsahoryi et al., 2020 ; Nchanji and Lutomia, 2021 ). Despite the growing body of research on coronavirus, agricultural production, and food security, few studies have attempted to conduct a thorough assessment of the literature and map the present level of scientific knowledge on the effect of coronavirus on agricultural production and food security (ECAP-FS). Hence, the goal of this research was to examine the effect of coronavirus on agricultural production and food security by employing bibliometric analyses techniques to recognise keywords in connection to two core aspects, namely the most prolific or productive writers and the most collaborative nations, and then to examine the strength of their association over the study period. The study characterised intellectual processes further by visualising and recognising the advancement of the co-citation network, cooperation network, and trends in ECAP-FS research. This research will not only aid in the identification of present research on ECAP-FS, but also contributes to an improved comprehension of the scientific knowledge of coronavirus and its impact on agricultural production, food security, and the investigation of its evolution via published papers included in the Web of Science (WoS) and Scopus databases. Because one database is unlikely to provide a comprehensive picture of knowledge and trends in a field, the authors recommend a four stage processes to achieve a merged database that integrates WoS and Scopus and then deletes identical publications using RStudio or R-package to perform author coupling, keywords co-occurrence network visualisation, university collaboration networks, and others using net2VOSviewer. This study will be among the few that explains how to integrate two datasets and utilise them to conduct different network associations in bibliometrix R-package (RStudio v.4.0.3 software).

Method and data collection

The scientometric technique was used to retrieve articles relating to the effect of coronavirus on agricultural production and food security. This method used resources from two different databases, WoS and Scopus, for the systematic reviews. Table 1 shows the eligibility and exclusion criteria that was used to access the relevant documents. The various steps employed in the review process were (databases, identification, screening, eligibility, merging, duplicate removal and included documents) (see Fig. 1 ). Processing and analysis of the data were then applied to the remaining documents. Scientometrics is defined as the research approach utilised in analysing and assessing science, innovation, and technology by applying statistics and quantitative analysis to explain the distribution and visualisation patterns of research within a specific nation, issue, field or institution (Orimoloye and Ololade, 2021 ). Scientometric evaluations have been used to analyse scientific trends and outputs, as well as the evolution of research, author productivity, journals, and nations, as well as to discover and measure international collaboration (Orimoloye and Ololade, 2021 ).

WoS: Web of Science.

WoS and Scopus were the two-database used for this study. WoS is a database collection administered by Thomson Reuters Institute of Scientific Information (ISI) that contains databases on humanities, social sciences, biology (i.e., Biosis), science (i.e., Core Collection) and computers (i.e., Inspec). WoS was previously the only and biggest accessible database for bibliometric analysis. However, Scopus that was launched by Elsevier, with ease of use in universities throughout the globe emerged as a key rival for doing such studies (Echchakoui, 2020 ). Scopus has the largest abstract and citation databases with over 22,800 journals from 5000 publishers worldwide was used in the review (Shaffril et al., 2018 ). Moreover, It is the most comprehensive interdisciplinary database of peer-reviewed literature in the social sciences, and is generally acknowledged and utilised for quantitative analyses (Guerrero-Baena et al., 2014 ).

Criteria for eligibility and exclusion

Various qualifying and exclusion criteria were considered. Title-based search for rapid visibility and retrieval was used. According to Ekundayo and Okoh ( 2018 ), a title-specific search offers the advantages of low loss, considerable retrieval, and sensitivity when compared to other types of searches such as a topic, field, or author search. First, concerning literature type, only journals and final articles were selected, which meant Article in Press, etc., were excluded. Secondly, non-English articles were excluded. Thirdly, a period of 6 years was used followed by the subject area, which focused on Environmental, Social, Agricultural, and Biological Sciences (Table 1 ) (Shaffril et al., 2018 ).

Systematic review process

To explore the current literature on ECAP-FS, we conducted a comprehensive literature review according to the rules provided by Tranfield et al. ( 2003 ). The systematic review process for this study involved four stages. The review process was performed in April 2021. The first stage was the selection of databases (WoS and Scopus). The second stage pinpointed keywords utilised for the search process. Based on prior research, keywords similar and related to the effect of COVID-19 on agricultural output and food security were used with a total of ( n  = 9, 421) published records found on WoS and Scopus, respectively (Table 2 ). The third stage was screening. Out of ( n  = 9, 421) papers eligible for evaluation at this stage, a total of ( n  = 7, 203) papers were excluded. The fourth stage was eligibility where the complete articles were accessible. Following a thorough review, a total of ( n  = 1, 46) publications were eliminated since some did not focus on the effect of coronavirus on agricultural production and food security. The fifth stage was merging the two documents ( n  = 6, 172 = 178). The sixth stage was the removal of duplicates ( n  = 4). The last round of evaluation yielded a total of ( n  = 174) papers for qualitative analysis (Fig. 1 ).

Processing and analysis of data

The research assessed data obtained for scientometric investigation utilising RStudio v.4.0.3 software with bibliometrix R-package and net2VOSviewer after reading the articles relevant to the study. The data were imported into RStudio, transformed to a bibliographic data frame, and normalised for duplicate matches (Aria and Cuccurullo, 2017 ; Ekundayo and Okoh, 2018 ). Net2VOSviewer (net,vos.path = NULL) embedded in RStudio v.4.0.3 software were used for visualisation. The VOSviewer programme created by Van Eck and Waltman ( 2009 ) is often used to visualise and evaluate a bibliometric network. Hamidah et al. ( 2021 ) and Zhang and Yuan ( 2019 ) made use of VOSviewer to analyse a bibliographic map on energy performance. Park and Nagy ( 2018 ) used VOSviewer to examine building control bibliographic data, and Van Eck and Waltman ( 2017 ) analysed citation-based clustering in the field of astronomy and astrophysics using VOSviewer. The research made use of Net2VOSviewer embedded in R studio to make visualisation maps, such as authors coupling, keyword co-occurrence network, and university collaboration network, based on bibliographic data. Each circle on the VOSviewer visual map represents a word. The term activity is represented by the circle and text size. The big circle and text show the chosen terms in a field. The distance between the two words reflects the degree of their association. In this case, the relationship between two words will be greater if the distance between them is small (Hamidah et al., 2021 ).

Web of Science and Scopus database merging for bibliometric analysis

The authors suggest the following four stage approach to combine the two databases shown in Fig. 1 and Table 3 .

As soon as required articles were sourced, we downloaded the documents separately from WoS and Scopus databases. For WoS, we clicked on export, which redirected us to another window where we selected “other file formats” under record content, and “BiTeX” under file format before we clicked export. For Scopus, we went to export document setting where we ticked all relevant boxes including “BibTeX” before clicking export. The second step was to transform (WoS.bib and Scopus.bib) to “bibtex” files. Here we used R or Rstudio software by loading the bibliometrix package “install.packages” (“bibliometrix”), and “library(bibliometrix)”, After that we specified the pathway using the command file1<- “path/savedrecs.bib” and file2 < - “path/scopus.bib” for WoS and Scopus files, respectively. After that we converted file (1&2) using command “f1<-convert2df(file1, dbsource = “isi”, format = “bibtex”)” and “f2<-convert2df(file2, dbsource = “scopus”, format = “bibtex”)” for WoS and Scopus respectively. We merged the two databases in R/Rstudio. For this operation to be successful, we used the command “j <-mergeDbSources(f1, f2, remove.duplicated = FALSE)”. Finally, the duplicate documents were removed using the command “M < -duplicatedMatching(j, Field = “TI”,tol = 0.95)”. We performed a bibliometric analysis for bibtex file in Rstudio, using Aria and Cuccurullo’s ( 2017 ) techniques and scripts in R, and utilising the net2VOSviewer for keywords co-occurrence network, collaboration networks of universities, authors coupling, amongst others.

Bibliometric analyses results

During the survey period, 174 papers were published on ECAP-FS; their characteristics are shown in Table 4 . The research had 851 authors, with a cooperation index of 5.1 and a document/author ratio of 0.20 (4.89 authors/document). Except for nine authors who published alone, all 842 authors were part of multi-author publications.

During the research period, an average of 6.0 citations per document were recorded. Lotka’s law scientific output for ECAP-FS study revealed a constant of 0.70 and beta coefficient of 3.88, with a Kolmogorov–Smirnoff goodness-of-fit of 0.94. Table 5 and Fig. 2 displays published research on ECAP-FS from 2016 to April 2021 in conjunction with the total citation of papers on average by year. The yearly pace of development was 56.64, with a mean overall of 12 ± 6, indicating that ECAP-FS research increased over time. This outcome agrees with the work of El Mohadebe et al. ( 2020 ) who stated that the number of published articles increased exponentially since the start of the COVID-19 pandemic. The rise in COVID-19 research reflects that it is a major danger to human health, the economy, and food security in industrialised and emerging nations (Carroll et al., 2020 ; Mottaleb et al., 2020 ; Alam and Khatun, 2021 ).

ATC/Y average total citations of articles published per year. NB: The yearly percentage rate of increase was 56.64.

During the survey period, research production varied, peaking in 2020 with 38.5% (67/174) of the total research output, followed by 2021 with 66 research articles accounting for 37.9% (66/174) during the same time. This result is liable to change when additional papers pertaining to ECAP-FS are published in 2021. The average total number of citations for published papers changed over time, peaking in 2016 (average = 11.8). Furthermore, the findings of this analysis identified the top 20 most prolific authors from 2016 to April 2021. Table 6 shows Gong B as the most productive author over the time, with six papers accounting for 3.45% of the total research publications on ECAP-FS. The following were placed second on the list: Baudron F, Peng W, and Zhang S who published three research articles each accounting for 1.7% of the total published research articles within the study period. The rest of the 17 authors published two articles within the same year. The quantity of a researcher’s academic output demonstrates their efficacy and propensity for conducting quality research (Orimoloye et al., 2021a )

Citation analysis reveals how many times a specific research article has been cited in other scientific articles. More cited research articles are considered significantly more influential than articles with fewer citations (Mishra et al., 2017 ; Nyam et al., 2020 ). Table 7 shows the top 20 papers on ECAP-FS in terms of citations in the field throughout the time. The list was compiled using the publications with the most citations (Echchakoui, 2020 ). In this research on ECAP-FS, Foyer et al. 2016 “Nature Plants” placed first with a total of 244 citations. Hart et al. 2018 “Functional Ecology” took second place with 60 citations, followed by Smiraglia D. 2016 “Environmental Research” with 52 citations during the same time period. Millar NS 2016 “Oecologia” and Tesfahunegn GB 2016 “Applied Geography” rated fourth and fifth with 43 and 42 citations, respectively. With 39, 23 and 21 citations, respectively, KC et al. 2018 “Plos One,” Pu and Zhong, 2020 “Global Food Security,” and Provenza FD 2019 “Frontiers in Nutrition” placed sixth, seventh, and eighth. As shown in Table 8 , the leading active writers were connected with institutions in both emerging and developed countries, including China (28), the United States (19), the United Kingdom (12), Italy (9), Spain (8), Australia (5), India (5), and Mexico (5). With the exception of China, the majority of the articles were from developed countries. China, the United States of America, United Kingdom, Italy, and Spain, among other countries, contributed the most articles in ECAP-FS, which is line with the work of Mottaleb et al. ( 2020 ). According to Orimoloye et al. ( 2021b ), research funding and scholarships have had a significant impact on the research output of many countries. As a result, this study indicates that economic assistance could help in the advancement of research in the area of ECAP-FS. Furthermore, during the research period, the total citation of published papers on average by each nation differed from one nation to another. Table 9 shows the top 20 citations by nation for ECAP-FS research papers. The data indicated that the most mentioned nations were industrialised ones, while China, a developing country, placed second among the most often referenced nations. The exceptional success of China research suggests that the nation performs well in sponsoring field research, possibly because the coronavirus originated in Wuhan City of China (Mottalab et al., 2020). Italy leads the way with 112 total citations and an average article citation of 12.44 for research papers published during the study duration, China was second with 107 citations and an average article citation of 3.82. During the same time period, the United States, the United Kingdom, Ethiopia, and Canada were placed third, fourth, fifth, and sixth, with total number of citations (average article citations) of 81 (4.26), 76 (6.33), 47 (23.50), and 40 (13.33), respectively.

This analysis also uncovered the most relevant sources for published academic research on ECAP-FS between 2016 and April 2021, as shown in Table 10 . Sustainability (Switzerland) was first with a total of 23 scientific papers on ECAP-FS. Agricultural Systems and Journal of Cleaner Production were ranked second and third with a total of 13 and 10 articles respectively. Global Food Security and Science of The Total Environment were rated fourth with eight articles each. Land was ranked fifth with five articles while Food Security, International Journal of Environmental Research and Public Health, Plos One were ranked sixth with four published articles each. Environmental Research and Journal of Integrative Agriculture rated seventh with three published articles on ECAP-FS throughout the review period.

Concerns are growing about the influence of COVID-19 on agricultural production, which could pose a significant threat to long-term food security and food supply (Pu and Zhong, 2020 ). Table 11 summarises the top 20 academics’ most relevant terms. In addition, Table 11 displays the most important keywords linked to ECAP-FS research, including keywords-plus (ID) as well as author keywords (DE). COVID-19, Food Security, Agriculture, Climate Change, Sustainable Development, Agricultural Production, Biodiversity, China, and Sustainability were among the nine keywords shared by keywords-plus (ID) and author keywords (DE). Eleven keywords were peculiar to authors’ keywords (Resilience, Ecosystem Services, Food Systems, COVID-19 Pandemic, Food Supply Chain, India, Land Take, Life Cycle Assessment, Nutrition, Conservation, and Dietary Diversity), and nine keywords were unique to keywords-Plus (Food Supply, Human, Article, Food Production, Land Use, Agricultural Robots, Agricultural Land, Controlled Study, and Cultivation). The distinct author keywords explicitly defined what COVID-19 affected as well as the means or elements engaged in the process (Nutrition, Dietary Diversity, Ecosystem Services, Resilience, Conservation, Food Systems, and Food Supply Chain of People). COVID-19 ( n  = 27, 15.5%), Food Security ( n  = 25, 14.4%), Agriculture ( n  = 18, 10.3%), Climate Change ( n  = 9, 5.2%), Sustainable Development ( n  = 5, 2.9%), Agricultural Production ( n  = 4, 2.3%), Biodiversity ( n  = 4, 2.3%), China ( n  = 4, 2.3%), COVID-19 Pandemic ( n  = 4, 2.3%) were author keyword phrases related with the detection of ECAP-FS.

The keyword analysis identified Food Security in 35 (20.1%) and 25 (14.4%) published papers by keyword-plus and author keyword, respectively, while Agricultural was found in 28 (16.1%) and 18 (10.3%) published papers by keyword-plus and author keyword, respectively. By author keyword and keyword-plus, Agricultural Production was detected in 4 (2.3%) and 28 (16.1%) publications, respectively. In the ECAP-FS study field, Climate Change was detected in 26 (14.9%) and 9 (5.2%) papers by keyword-plus and author keyword, respectively. The review indicates that research on ECAP-FS emphasised these agricultural-related issues several times, implying that COVID-19 has an effect on agriculture, agricultural production, sustainable development, food security, and food supply of the general public, which is exacerbated by climate change, and is a major danger to food security, economy and human health (Mottaleb et al., 2020 ).

The connection between influential authors, keywords, journals, and trending topics was investigated using co-citation network analysis (Leydesdorff, 2009 ). Articles are said to be co-cited when they are cited and appear in other publications’ reference lists (Nyam et al., 2020 ). The top 20 authors coupling in Fig. 3 explains the authors coupling on ECAP-FS-related research. Every node in the network symbolises a distinct author who is linked to others. Connecting lines reflect author-to-author linking routes. The number of lines from each node correlates to the number of published papers that referenced the writer. The cluster of authors network, which comprises 20 nodes (authors), has no less than 18 interconnections. Other indicators of often expressed ideas and frameworks linked to ECAP-FS include nation collaboration (Fig. 4 ) and university collaboration network (Fig. 5 ).

The top 20 authors coupling on agricultural production and food security published articles. (Every node in the network symbolises a distinct author who is linked to others. Connecting lines reflect author-to author linking routes).

The top 27 nation collaboration networks on agricultural production and food security. (Each node represents a country, and the lines represent their collaboration).

The top 20 university collaboration networks on agricultural production and food security research.

Authors with multiple affiliations have made significant contributions to nation and university collaborative networks (Figs. 4 and 5 ). Our findings indicated that studies on ECAP-FS were conducted at institutions in both advanced and developing nations between 2016 and April 2021. The Wageningen University (Netherland), the China Agricultural University (China), the Zhejiang University (Asia), and University of Pretoria (South Africa) had the greatest collaboration network on ECAP-FS studies followed by the University of Western Australia (Australia), University of Leeds (UK), University of Alberta (Canada), University of Sydney (Australia), Case Western Reserve University (USA), Chinese University of Hong Kong (China) and the International Crop Research Institute. The University of Oxford was the only university that did not collaborate with any of the universities during the study period. Figure 4 depicts the networks of collaboration on ECAP-FS for 27 countries. The number of collaboration paths varied from one to 17. The number of partnerships was highest in the USA ( n  = 17), followed by China (n = 10), Australia ( n  = 8), the United Kingdom ( n  = 8), Canada ( n  = 5), the Netherlands ( n  = 4), Germany ( n  = 4), South Africa ( n  = 4), Uganda ( n  = 3), India ( n  = 3), Malaysia ( n  = 2), Denmark ( n  = 2), France ( n  = 2), Spain ( n  = 2), and New Zealand ( n  = 2). The remaining nations had one collaboration network. This outcome is consistent with El Mohadab et al. ( 2020 ) as the analysis of a nation’s collaboration is a vital type of analysis, because it allows for the visualisation of the most influential nations in a given field of research, revealing the level of scientific cooperation between the countries. The following network colour codes were prominent: light green for the USA network; light blue for the China network; purple for the Australia network; orange for the United Kingdom network; and brown for the Spain network.

Figure 6 depicts the top 30 keywords of co-occurrence network, the related visualisation and the association strength of ECAP-FS. The co-occurrence of author keywords was examined to illustrate the research hotspots in ECAP-FS. The threshold for keyword co-occurrence was set at 10, and 30 keywords out of 708 were categorised as visualisation elements. The distance between the components of each pairings indicated topic similarity and relative strength. Individual term clusters were allocated different colours of circles. The network in Fig. 6 depicts three different clusters, each reflecting a branch of research in the ECAP-FS literature. The number of publications in which the keywords co-occurred was shown by the connections between specific keywords. The main themes with the highest overall connection strength in the ECAP-FS literature were COVID-19, Food Security, Agriculture, and Climate Change.

The co-occurrence network visualisation of 30 keywords and their relationship strength of agricultural production and food security research.

The ECAP-FS scientific field has three subfields (clusters of author keywords), which are as follows:

The blue cluster includes terms such as COVID-19, Food Supply, Food Production, China, Food Security, and Agricultural Production.’

The red cluster grouped the keywords Agricultural Land, Catering Services, Environmental Protection, Humans, Meat, Human, Food Industry, Article, Female, Priority Journal, Procedures, Controlled Study, and Environmental Sustainability.

The green cluster grouped the keywords Economic and Social Effects, Agriculture, Agricultural Robots, Sustainable Development, Climate Change, Land Use, Greenhouse Gases, Ecosystem, and Biodiversity. The findings revealed a significant variation in the co-occurrence of author keywords in individual articles in the ECAP-FS literature. This demonstrated the scientific field’s multifaceted and multidimensional nature. This result is agreement with the work of Orimoloye et al. ( 2021b ).

Figure 7 depicts the frequency of word occurrence of the top 70 most utilised title keywords in ECAP-FS studies. During the research, a word cloud was generated using the titles of published articles that contained the most frequently used keywords in ECAP-FS research. This revealed the most commonly used word or phrase in ECAP-FS research. Within the word cloud on ECAP-FS research, various regions of connections and the most significant words used were determined. For example, COVID-19, food security, agriculture, climate change, ecosystem services, resilience, agricultural production, sustainable development, food system, and China were recognised as the most prevalent or prominent themes in ECAP-FS studies.

Word cloud or frequency of word occurrence of the top 70 most often used title keywords in agricultural production and food security research.

The COVID-19 pandemic has received significant recognition since the outbreak, and serious effort has been expended by researchers around the world in various fields. The present bibliometric analysis of COVID-19 examined the resulting effects on agricultural production and food security research trends from 2016 to April 2021 by means of data acquired from WoS and Scopus. According to our findings in ECAP-FS, there has been an exponential rise in research publications. This indicates that studies on ECAP-FS received increasing attention during last few years especially in 2020 and 2021, most likely due to COVID-19 pandemic related research by authors from different counties of the world like China, USA and the United Kingdom. Furthermore, most of the productive authors in ECAP-FS at the time of this research were from China, possibly because the pandemic was first discovered in Wuhan City.

The findings of this analysis revealed that few articles came from Africa. In terms of country and institution collaboration networks, few of the countries and institutions collaborated with the countries in Africa except for the University of Pretoria, which had a strong collaboration network on ECAP-FS research during the period of study. According to the word cloud analysis and frequency analysis of the frequently used keywords and keyword-plus demonstrated that the most topical issues in ECAP-FS are COVID-19, food security, agriculture, climate change, agricultural production, sustainable development, biodiversity and sustainability. These results demonstrated the most persistent issues related to ECAP-FS; this was buttressed by another conceptual framework indicator such as keyword co-occurrence networks.

The bibliometric survey performed in this study has some limitations, such as the use of two databases (Scopus and WoS), the strictness of the search keywords and search approach employed, as well as the exclusion of other document types (e.g., conference papers, books chapters, reviews, abstracts, meetings and notes, etc.) and published articles in languages other than English (French, Dutch, Chinese). Despite the limitations, this research seems to be the first bibliometric analysis on ECAP-FS-related studies, which adds to the evidence base and will drive further studies. Furthermore, WoS and Scopus have greater coverage than other databases, dependable indexing technology that reduces the “indexer effect,” and are highly regarded by scientific communities. Other databases, such as ScienceDirect, Education Resource Information Center (ERIC), and Directory of Open Access Journals (DOAJ), should be evaluated in future studies.

Data availability

All data analysed are contained in the paper.

Alam GM, Khatun MN (2021) Impact of COVID-19 on vegetable supply chain and food security: empirical evidence from Bangladesh. PLoS ONE 16(3):e0248120

Article   CAS   Google Scholar  

Aria M, Cuccurullo C (2017) bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetr 11(4):959–975 https://doi.org/10.1016/j.joi.2017.08.007

Article   Google Scholar  

Bai HM (2020) The Socio-economic implications of the Coronavirus pandemic (COVID-19) A Review. ComFin Res 8(4):8–17

Carroll N, Sadowski A, Laila A, Hruska V, Nixon M, Ma DWL, Haines J (2020) The Impact of COVID-19 on Health Behavior, Stress, Financial and Food Security among Middle to High Income Canadian Families with Young Children. Nutrients. 12:2352 https://doi.org/10.3390/nu12082352

Article   CAS   PubMed Central   Google Scholar  

Ceballos F, Kannan S, Kramer B (2020) Impacts of a national lockdown on smallholder farmers’ income and food security: empirical evidence from two states in India. World Dev 136:105069. https://doi.org/10.1016/j.worlddev.2020.105069

Dev SM (2020) Addressing COVID-19 impacts on agriculture, food security, and livelihoods in India, IFPRI book chapters, in: COVID-19 and global food security, ch. 7, International Food Policy Research Institute (IFPRI), p 33–35

Devereux S, Béné C, Hoddinott J (2020) Conceptualising COVID-19’s impacts on household food security. Food Secur 12(4):769–772

Echchakoui S (2020) Why and how to merge Scopus and Web of Science during bibliometric analysis: the case of sales force literature from 1912 to 2019. J Market Anal 8(3):165–184. https://doi.org/10.1057/s41270-020-00081-9

Ekundayo TC, Okoh AI (2018) A global bibliometric analysis of Plesiomonas-related research (1990–2017). PLoS ONE 13(11):1–18. https://doi.org/10.1371/journal.pone.0207655

El Mohadab M, Bouikhalene B, Safi S (2020) Bibliometric method for mapping the state of the art of scientific production in Covid-19. Chaos Solitons Fractals 139:110052

Article   MathSciNet   Google Scholar  

Elsahoryi N, Al-sayyed H, Odeh M, Mcgrattan A, Hammad F (2020) Effect of Covid-19 on food security: a cross-sectional survey. Clin Nutr ESPEN 40:171–178. https://doi.org/10.1016/j.clnesp.2020.09.026

Article   PubMed   PubMed Central   Google Scholar  

Food and Agricultural Organisation (2020) Sustainable crop production and COVID-19. https://doi.org/10.4060/ca8807en

Foyer CH, Lam HM, Nguyen HT, Siddique KH, Varshney RK, Colmer TD, Considine MJ (2016) Neglecting legumes has compromised human health and sustainable food production. Nat Plants 2(8):1–10

Genkin AS, Mikheev AA (2020) Influence of coronavirus crisis on food industry economy. Foods Raw Mater 8: 2

Gong B (2018) Agricultural reforms and production in China: changes in provincial production function and productivity in 1978–2015 Stochastic frontier analysis. Semi-varying coefficient model China’s agricultural production and productivity. Rural reforms multi-segment industry. https://doi.org/10.1016/j.jdeveco.2017.12.005

Gong B, Zhang S, Liu X, Chen KZ (2021) The Zoonotic diseases, agricultural production, and impact channels: evidence from China. Global Food Secur28:100463. https://doi.org/10.1016/j.gfs.2020.100463

Gregorio GB, Ancog RC (2020) Assessing the impact of the COVID-19 pandemic on agricultural production in Southeast Asia. Toward Transform Change Agric Food Syst 17(1):1–14

Google Scholar  

Guerrero-Baena MD, Gómez-Limón JA, Fruet Cardozo JV (2014) Are multi-criteria decision-making techniques useful for solving corporate finance problems? A bibliometric analysis. Rev de Metod Cuantitativos para la Econ y la Empres 17:60–79

Hamidah I, Pawinanto RE, Mulyanti B, Yunas J (2021) A bibliometric analysis of micro electromechanical system energy harvester research. Heliyon 7(3):e06406. https://doi.org/10.1016/j.heliyon.2021.e06406

Hanna R, Xu Y, Victor DG (2020) After COVID-19, green investment must deliver jobs to get political traction. Nature 582:178–180

Article   ADS   CAS   Google Scholar  

Hart MM, Antunes PM, Chaudhary VB, Abbott LK (2018) Fungal inoculants in the field: Is the reward greater than the risk? Funct Ecol 32(1):126–135

KC KB, Dias GM, Veeramani A, Swanton CJ, Fraser D, Steinke D, Fraser ED (2018) When too much isn’t enough: does current food production meet global nutritional needs? PLoS ONE 13(10):e0205683

Laborde D, Martin W, Vos R (2020a) Poverty and food insecurity could grow dramatically as COVID-19 spreads. International Food Policy Research Institute (IFPRI), Washington

Book   Google Scholar  

Laborde D, Martin W, Swinnen J, Vos R (2020b) COVID-19 risks to global food security. Science 369(6503):500–502

Leydesdorff L (2009) How are new citation‐based journal indicators adding to the bibliometric toolbox? J Am Soc Inf Sci Technol 60(7):1327–1336

Mishra D, Luo Z, Jiang S, Papadopoulos T, Dubey R (2017) A bibliographic study on big data: concepts, trends and challenges. Bus Process Manag J 23:555–573. https://doi.org/10.1108/BPMJ-10-2015-0149

Mottaleb KA, Mainuddin M, Sonobe T (2020) COVID-19 induced economic loss and ensuring food security for vulnerable groups: policy implications from Bangladesh. PLoS ONE 15(10):e0240709. https://doi.org/10.1371/journal.pone.0240709

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nchanji EB, Lutomia CK (2021) Regional impact of COVID-19 on the production and food security of common bean smallholder farmers in Sub-Saharan Africa: implication for SDG’s. Global Food Secur 29:100524

Nechifor V, Priscila M, Ferrari E, Laichena J, Kihiu E, Omanyo D, Musamali R, Kiriga B (2021) Food security and welfare changes under COVID-19 in Sub-Saharan Africa: impacts and responses in Kenya. Global Food Secur 28:100514. https://doi.org/10.1016/j.gfs.2021.100514

Nyam YS, Kotir JH, Jordaan AJ, Ogundeji AA, Adetoro AA, Orimoloye IR (2020) Towards understanding and sustaining natural resource systems through the systems perspective: a systematic evaluation. Sustainability 12(23):9871

Orimoloye IR, Ololade OO (2021) Global trends assessment of environmental health degradation studies from 1990 to 2018. Environ Dev Sustainabil 23(3):3251–3264

Orimoloye IR, Belle JA, Olusola AO, Busayo ET, Ololade OO (2021a) Spatial assessment of drought disasters, vulnerability, severity and water shortages: a potential drought disaster mitigation strategy. Nat Hazards 105(3):2735–2754

Orimoloye IR, Belle JA, Ololade OO (2021b) Exploring the emerging evolution trends of disaster risk reduction research: a global scenario. Int J Environ Sci Technol 18(3):673–690

Park JY, Nagy Z (2018) Comprehensive analysis of the relationship between thermal comfort and building control research—a data-driven literature review. Renew Sustain Energy Rev 82:2664–2679

Petetin L (2020) The COVID-19 crisis: an opportunity to integrate food democracy into post-pandemic food systems. Eur J Risk Regul 11(2):326–336

Pu M, Zhong Y (2020) Rising concerns over agricultural production as COVID-19 spreads: lessons from China. Global Food Secur 26:100409. https://doi.org/10.1016/j.gfs.2020.100409

Shaffril HAM, Krauss SE, Samsuddin SF (2018) A systematic review on Asian’s farmers’ adaptation practices towards climate change. Sci Total Environ 644:683–695. https://doi.org/10.1016/j.scitotenv.2018.06.349

Article   ADS   CAS   PubMed   Google Scholar  

Singh S, Kumar R, Panchal R, Tiwari MK (2021) Impact of COVID-19 on logistics systems and disruptions in food supply chain. Int J Prod Res 59(7):1993–2008. https://doi.org/10.1080/00207543.2020.1792000

Tamru S, Hirvonen K, Minten B (2020) Impacts of the COVID-19 crisis on vegetable value chains in Ethiopia. IFPRI book chapters, International Food Policy Research Institute (IFPRI) pp 81–83. https://doi.org/10.2499/p15738coll2.133762_18

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222. https://doi.org/10.1111/1467-8551.00375

Van Eck NJ, Waltman L (2017) Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics 111(2):1053–1070. https://doi.org/10.1007/s11192-017-2300-7

Van Eck NJ, Waltman L (2009) VOSviewer: a computer program for bibliometric mapping. In: Larsen B, & Leta J (eds) 12th International conference on Scientometrics and Informetrics, ISSI 2009, pp. 886–897

Xu Z, Elomri A, El Omri A, Kerbache L, Liu H (2021) The compounded effects of COVID-19 pandemic and desert locust outbreak on food security and food supply chain. Sustainability 13(3):1063

Zhang S, Wang S, Yuan L, Liu X, Gong B (2020) The impact of epidemics on agricultural production and forecast of COVID-19. China Agric Econ Rev Emerald Group Publishing 12(3):409–425

Zhang W, Yuan H (2019) A bibliometric analysis of energy performance contracting research from 2008 to 2018. Sustainability 11(13):3548

Download references

Author information

Authors and affiliations.

Department of Agricultural Economics, University of the Free State, Bloemfontein, South Africa

Collins C. Okolie & Abiodun A. Ogundeji

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Abiodun A. Ogundeji .

Ethics declarations

Competing interests.

The author declares no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Okolie, C.C., Ogundeji, A.A. Effect of COVID-19 on agricultural production and food security: A scientometric analysis. Humanit Soc Sci Commun 9 , 64 (2022). https://doi.org/10.1057/s41599-022-01080-0

Download citation

Received : 09 September 2021

Accepted : 08 February 2022

Published : 28 February 2022

DOI : https://doi.org/10.1057/s41599-022-01080-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Integrated modelling of the determinants of household food insecurity during the 2020–2021 covid-19 lockdown in uganda.

• Henry Musoke Semakula
• Jovia Nakato

Agriculture & Food Security (2024)

Prediction of impacts and outbreak of COVID-19 on the society using distinct machine learning algorithms

• Taspia Tazri Chaity
• Md. Ashikur Rahman Khan
• Fardowsi Rahman

Iran Journal of Computer Science (2024)

Mapping scientific knowledge discovery on COVID-19 pandemic and agriculture: a bibliometric analysis and future research directions

Environmental Science and Pollution Research (2023)

Circular economy and agriculture: mapping scientific productivity, research pattern and future research direction

• Tabassum Ali

Environment, Development and Sustainability (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies