about the world
Seven dimensions evaluated for the 3 different components of each C.
Aspects of the overall educational program teaching, emphasizing, and promoting the 4Cs | |
Availability and access to different means, materials, space, and expertise, digital technologies, mnemonic and heuristic methods, etc. to assist in the proper use and exercise of the 4Cs | |
Actual student and program use of available resources promoting the 4Cs | |
Critical reflection and metacognition on the process being engaged in around the 4Cs | |
The formal and informal training, skills, and abilities of teachers/trainers and staff and their program of development as promoters of the 4Cs | |
Use and integration of the full range of resources external to the institution available to enhance the 4Cs | |
Availability of resources for students to create and actualize products, programs, events, etc. that require the exercise, promotion, or manifestation of the 4Cs |
* Educational-level dependent and potentially less available for younger students or in some contexts.
The grid itself can be used in several important and different ways by different educational stakeholders: (1) by the institution itself in its self-evaluation and possible preparation for a certification or labelization process, (2) as an explicit list of criteria for external evaluation of the institution and its 4Cs-related programs, and (3) as a potential long-term development targeting tool for the institution or the institution in dialogue with the labelization process.
Dropping the component of “creative person” that is not relevant at the institutional level, this evaluation grid is based on Rhodes’ ( 1961 ) classic “4P” model of creativity, which remains the most concise model today ( Lubart and Thornhill-Miller 2019 ). The three “P” components retained are: creative process , creative environment , and creative product . Creative process refers to the acquisition of a set of tools and techniques that students can use to enhance the creativity of their thinking and work. Creative environment (also called “Press” in earlier literature) is about how the physical and social surroundings of students can help them be more creative. Finally, creative product refers to the evaluation of actual “productions” (e.g., a piece of art, text, speech, etc.) generated through the creative process.
Our evaluation grid divides critical thinking into three main components: critical thinking about the world , critical thinking about oneself (self-reflection), as well as critical action and decision making . The first component refers to having an evidence-based view of the exterior world, notably by identifying and evaluating sources of information and using them to question current understandings and solve problems. Self-reflection refers to thinking critically about one’s own life situation, values, and actions; it presupposes the autonomy of thought and a certain distance as well as the most objective observation possible with regard to one’s own knowledge (“meta-cognition”). The third and final component, critical action and decision making, is about using critical thinking skills more practically in order to make appropriate life decisions as well as to be open to different points of view. This component also addresses soft skills and attitudes such as trusting information.
Our evaluation framework for critical thinking was in part inspired by Barnett’s “curriculum for critical being” (2015), whose model distinguishes two axes: one defined by the qualitative differences in the level of criticality attained and the second comprised of three different domains of application: formal knowledge, the self, and the world. The first two components of our framework (and the seven dimensions on which they are rated) reflect and encompass these three domains. Similar to Barrett’s proposal, our third rubric moves beyond the “skills-plus-dispositions” model of competency implicit in much theorizing about critical thinking and adds the importance of “action”—not just the ability to think critically and the disposition to do so, but the central importance of training and practicing “critical doing” ( Barnett 2015 ). Critical thinking should also be exercised collectively by involving students in collective thinking, facilitating the exchange of ideas and civic engagement ( Huber and Kuncel 2016 ).
The first component of collaboration skills in the IICD grid is engagement and participation , referring to the active engagement in group work. Perspective taking and openness concerns the flexibility to work with and accommodate other group members and their points of view. The final dimension— social regulation —is about being able to reach for a common goal, notably through compromise and negotiation, as well as being aware of the different types of roles that group members can hold ( Hesse et al. 2015 ; Rusdin and Ali 2019 ; Care et al. 2016 ). (These last two components include elements of leadership, character, and emotional intelligence as sometimes described in other soft-skill and competency-related systems.) Participation, social regulation, and perspective taking have been identified as central social skills in collaborative problem solving ( Hesse et al. 2015 ). Regarding social regulation in this context, recognizing and profiting from group diversity is key ( Graesser et al. 2018 ). When describing an assessment in an educational setting of collaborative problem solving (with a task in which two or more students have to collaborate in order to solve it, each using a different set of resources), two main underpinning skills were described for the assessment: the social skill of audience awareness (“how to adapt one’s own behavior to suit the needs of the task and the partner’s requirements”, Care et al. 2016, p. 258 ) and the cognitive skill of planning and executing (developing a plan to reach for a goal) ( Care et al. 2016 ). The former is included in the perspective taking and openness rubric and the latter in the social regulation component in the IICD grid. Evans ( 2020 ) identified four main collaboration skills consistently mentioned in the scientific literature that are assessed in the IICD grid: the ability to plan and make group decisions (example item from the IICD grid: teachers provide assistance to students to overcome differences and reach a common goal during group work); the ability to communicate about thinking with the group (assessed notably in the meta-reflection strand of the IICD grid); the ability to contribute resources, ideas, and efforts and support group members (included notably in the engagement and participation as well as the social regulation components); and finally, the ability to monitor, reflect, and adapt individual and group processes to benefit the group (example item from the IICD grid: students use perspective-taking tools and techniques in group activities).
The evaluation grid for communication is also composed of three dimensions: message formulation, message delivery, and message and communication feedback . Message formulation refers to the ability to design and structure a message to be sent, such as outlining the content of an argument. Message delivery is about effectively transmitting verbal and non-verbal aspects of a message. Finally, message and communication feedback refers to the ability of students and teachers to understand their audience, analyze their social surroundings, and interpret information in context. Other components of communication skills such as theory of mind, empathy, or emotional intelligence are also relevant and included in the process of applying the grid. Thompson ( 2020 ) proposes a four-component operationalized definition of communication for its assessment in students. First, they describe a comprehension strand covering the understanding and selection of adequate information from a range of sources. Message formulation in the IICD grid captures this dimension through its focus on content analysis and generation. Second, the presentation of information and ideas is mentioned in several different modes, adjusted to the intended audience, verbally as well as non-verbally. The message delivery component of the IICD grid focuses on these points. Third, the authors note the importance of communication technology and its advanced use. The IICD grid also covers the importance of technology use in its tools and techniques category, with, for example, an item that reads: students learn to effectively use a variety of formats of communication (social media, make a video, e-mail, letter writing, creating a document). Finally, Thompson ( 2020 ) describes the recognition of cultural and other differences as an important aspect of communication. The IICD grid aims at incorporating these aspects, notably in the meta-reflection category under each of the three dimensions.
5.1. the 4cs in informal educational contexts.
So far, the focus has been on rather formal ways of nurturing the 4Cs. Although institutions and training programs are perhaps the most significant and necessary avenues of education, they are not the sole context in which 4Cs’ learning and improvement can manifest. One other important potential learning context is game play. Games are activities that are present and participated in throughout human society—by those of all ages, genders, and socio-economic statuses ( Bateson and Martin 2013 ; Huizinga 1949 ; Malaby 2007 ). This informal setting can also provide favorable conditions to help improve the 4Cs ( van Rosmalen et al. 2014 ) and should not be under-appreciated. Games provide a unique environment for learning, as they can foster a space to freely explore possibilities and one’s own potential ( de Freitas 2006 ). We argue that games are a significant potential pathway for the improvement of the 4Cs, and as such, they merit the same attention as more formal ways of learning and developing competencies.
Compared to schools and educational institutions, the focus of IICD’s evaluation framework for games (see International Institute for Competency Development 2021 ) is more narrow. Thus, it is fundamentally different from the institutional grid: games, complex and deep as they can sometimes be, cannot directly be compared to the complexity of a school curriculum and all the programs it contains. The evaluation of a game’s effectiveness for training/improving a given C rests on the following principle: if a game presents affordances conducive to exercising a given skill, engaged playing of that game should help improve that skill.
The game’s evaluation grid is scored based on two criteria. For example, as a part of a game’s rating as a tool for the development of creativity, we determine the game must first meet two conditions. First, whether or not the game allows the opportunity for creativity to manifest itself: if creativity cannot occur in the game, it is obviously not eligible to receive ratings for that C. Second, whether or not creativity is needed in order to perform well in the game: if the players can win or achieve success in the game without needing creativity, this also means it cannot receive a rating for that C. If both conditions are met, however, the game will be considered potentially effective to improve creativity through the practice of certain components of creative behavior. This basic principle applies for all four of the Cs.
As outlined in Table 3 , below, the evaluation grid for each of the four Cs is composed of five components relevant to games that are different for each of the Cs. The grid works as follows: for each of the five components of each C, we evaluate the game on a list of sub-components using two yes/no scales: one for whether it is “possible” for that subcomponent to manifest and one for whether that sub-component is “required for success” in the game. This evaluation is done for all sub-components. After this, each general component is rated on the same two indicators. If 60% (i.e., three out of five) or more sub-components are positively rated as required, the general component is considered required. Then, the game is evaluated on its effectiveness for training and improving each of the 4Cs. If 60% or more components are positively rated as required, the game will be labelized as having the potential to be effective for training and improving the corresponding C.
Five different components evaluated for each C by the 4Cs assessment framework for games.
Originality | Divergent Thinking | Convergent Thinking | Mental Flexibility | Creative Dispositions | |
Goal-adequate judgment/ discernment | Objective thinking | Metacognition | Elaborate eeasoning | Uncertainty management | |
Collaboration fluency | Well-argued deliberation and consensus-based decision | Balance of contribution | Organization and coordination | Cognitive syncing, input, and support | |
Social Interactions | Social cognition | Mastery of written and spoken language | Verbal communication | Non-verbal communication |
The evaluation grid for creativity is based on the multivariate model of creative potential (see Section 2.1.1 and Lubart et al. 2013 for more information) and is composed of four cognitive factors and one conative factor: originality , divergent thinking , convergent thinking , mental flexibility , and creative dispositions . Originality refers to the generation of ideas that are novel or unexpected, depending on the context. Divergent thinking corresponds to the generation of multiple ideas or solutions. Convergent thinking refers to the combination of multiple ideas and the selection of the most creative idea. Mental flexibility entails changing perspectives on a given problem and breaking away from initial ideas. Finally, creative dispositions concerns multiple personality-related factors conducive to creativity, such as openness to experience or risk taking.
The evaluation grid for critical thinking echoes Halpern’s ( 1998 ) as well as Marin and Halpern’s ( 2011 ) considerations for teaching this skill, that is, taking into consideration thinking skills, metacognition, and dispositions. The five components of the critical thinking grid are: goal-adequate discernment, objective thinking, metacognition, elaborate reasoning, and uncertainty management. Goal-adequate discernment entails the formulation of inferences and the discernment of contradictions when faced with a problem. Objective thinking corresponds to the suspension of one’s own judgment and the analysis of affirmations and sources in the most objective manner possible. Metacognition, here, is about questioning and reassessing information, as well as the awareness of one’s own cognitive biases. Elaborate reasoning entails reasoning in a way that is cautious, thorough, and serious. Finally, uncertainty management refers to the dispositional propensity to tolerate ambiguity and accept doubt.
The evaluation grid for collaboration is based on the quality of collaboration (QC) method ( Burkhardt et al. 2009 ; see Section 2.4.2 for more details) and is composed of the following five components: collaboration fluidity, well-argued deliberation and consensus-based decision, balance of contribution, organization and coordination, and cognitive syncing, input, and support. Collaboration fluidity entails the absence of speech overlap and the presence of a good flow in terms of turns to speak. Well-argued deliberation and consensus-based decision is about contributing to the discussion and task at hand, as well as participating in discussions and arguments, in order to obtain a consensus. Balance of contribution refers to having equal or equivalent contributions to organization, coordination, and decision making. Organization and coordination refers to effective management of roles, time, and “deadlines”, as well as the attribution of roles depending on participants’ skills. Finally, cognitive syncing, input, and support is about bringing ideas and resources to the group, as well as supporting and reinforcing other members of the group.
The five components used to evaluate communication in games include both linguistic, pragmatic, and social aspects. Linguistic skills per se are captured by the mastery of written and spoken language component. This component assesses language comprehension and the appropriate use of vocabulary. Pragmatic skills are captured by the verbal and non-verbal communication components and refer to the efficient use of verbal and body signals in the context of the game to achieve one’s communicative goals ( Grassmann 2014 ; Matthews 2014 ). Finally, the grid also evaluates social skills with its two last components, social interactions and social cognition, which, respectively, refer to the ability to interact with others appropriately—including by complying with the rules of the game—and to the understanding of other people’ mental states ( Tomasello 2005 ).
Each of the 4Cs is a broad, multi-faceted concept that is the subject of a tremendous amount of research and discussion by a wide range of stakeholders in different disciplines, professions, and parts of the educational establishment. The development of evaluation frameworks to allow support for the 4Cs to be assessed and publicly recognized, using a label, is an important step for promoting and fostering these skills in educational contexts. As illustrated by IICD’s 4Cs Framework for educational institutions and programs, as well as its games/activities evaluation grid, the specific criteria to detect support for each C can vary depending upon the educational context (e.g., formal and institutional level or informal and at the activity level). Yet considering the 4Cs together highlights some additional observations, current challenges, and opportunities for the future that are worthy of discussion.
One very important issue for understanding the 4Cs and their educational implementation that can be simultaneously a help and a hindrance for teaching them—and also a challenge when assessing them—is their multidimensionality and interrelatedness. In other words, the 4Cs are not entirely separate entities but instead, as Figure 2 shows, should be seen as four interlinked basic “elements” for future-oriented education that can help individuals in their learning process and, together, synergistically “bootstrap” the development of their cognitive potentials. Lamri and Lubart ( 2021 ), for example, found a certain base level of creativity was a necessary but not sufficient condition for success in managerial tasks, but that high-level performance required a combination of all four Cs. Some thinkers have argued that one cannot be creative without critical thinking, which also requires creativity, for example, to come up with alternative arguments (see Paul and Elder 2006 ). Similarly, among many other interrelationships, there is no collaboration without communication—and even ostensibly individual creativity is a “collaboration” of sorts with the general culture and precursors in a given field. As a result, it ranges from impossible to suboptimal to teach (or teach towards) one of the 4Cs without involving one or more of the others, and this commingling also underscores the genuine need and appropriateness of assessing them together.
“‘Crea-Critical-Collab-ication’: a Dynamic Interactionist Model of the 4Cs”. (Illustration of the interplay and interpenetration of creativity, critical thinking, collaboration, and communication shown in dimensional space according to their differing cognitive/individual vs. social/interpersonal emphases; (© 2023, Branden Thornhill-Miller. All Rights Reserved. thornhill-miller.com; accessed on 20 January 2023)).
From this perspective, Thornhill-Miller ( 2021 ) proposed a “dynamic interactionist model of the 4Cs” and their interrelated contributions to the future of education and work. Presented in Figure 2 , this model is meant to serve as a visual and conceptual aid for understanding the 4Cs and their interrelationships, thereby also promoting better use and understanding of them in pedagogical and policy settings. In addition to suggesting the portmanteau of “crea-critical thinking” as a new term to describe the overlap of much of the creative and critical thinking processes, the title of this model, “Crea-Critical-Collab-ication”, is a verbal representation of the fluid four-way interrelationship between the 4Cs visually represented in Figure 2 (a title meant to playfully repackage the 4Cs for important pedagogical and policy uses). This model goes further to suggest some dimensional differences in emphases that, roughly speaking, also often exist among the 4Cs: that is to say, the frequently greater emphasis on cognitive or individual elements at play in creativity and critical thinking in comparison to the social and interpersonal aspects more central to communication and collaboration ( Thornhill-Miller 2021 ).
Similarly focused on the need to promote a phase change towards future-oriented education, Lucas ( 2019 ) and colleagues have suggested conflating creative thinking and critical thinking in order to propose “3Cs” (creative thinking, communication, and collaboration) as new “foundational literacies” to symmetrically add to the 3Rs (Reading, wRiting, and aRithmetic) of previous educational eras. Although we applaud these efforts, from our applied research perspective, we believe that the individual importance of, and distinct differences between, creative thinking and critical thinking support preserving them both as separate constructs in order to encourage the greatest development of each of them. Moreover, if only three categories were somehow required or preferable, one could argue that uniting communication and collaboration (as “collab-ication” suggests) might be preferable—particularly also given the fact that substantial aspects of communication are already covered within the 3Rs. In any case, we look forward to more such innovations and collaborations in this vibrant and important area of work at the crossroads between research, pedagogy, and policy development.
The rich literature in each of the 4Cs domains shows the positive effects of integrating these dimensions into educational and professional curricula. At the same time, the complexity of their definitions makes them difficult to assess, both in terms of reliability (assessment must not vary from one measurement to another) and of validity (tests must measure that which they are intended to measure). However, applied research in this area is becoming increasingly rigorous, with a growing capacity to provide the necessary tools for evidence-based practice. The development of these practices should involve interdisciplinary teams of teachers and other educational practitioners who are equipped and trained accordingly. Similarly, on the research side, further exploration and clarification of subcomponents of the 4Cs and other related skills will be important. Recent efforts to clarify the conceptual overlap and hierarchical relations of soft skills for the future of education and work, for example, have been helpful and promising (e.g., Joie-La Marle et al. 2022 ; Lamri et al. 2022 ). But the most definitive sort of taxonomy and measurement model that we are currently lacking might only be established based on the large-scale administration of a comprehensive battery of skill-measuring psychometric tests on appropriate cross sections of society.
The rapid development and integration of new technologies will also aid and change the contexts, resources, and implementation of the 4Cs. For example, the recent developments make it clear that the 4Cs will be enhanced and changed by interaction with artificially intelligence, even as 4Cs-related skills will probably, for the same reason, increasingly constitute the core of available human work in the future (see, e.g., Ross 2018 ). Similarly, research on virtual reality and creativity suggest that VR environments assist and expand individual and collaborative creativity ( Bourgeois-Bougrine et al. 2022 ). Because VR technologies offer the possibility of enhanced and materially enriched communication, collaboration, and information availability, they not only allow for the enhancement of creativity techniques but also for similar expansions and improvements on almost all forms of human activity (see Thornhill-Miller and Dupont 2016 )—including the other three Cs.
Traditional educational approaches cannot meet the educational needs of our emergent societies if they do not teach, promote, and assess in line with the new learner characteristics and contexts of the 21st century ( Sahin 2009 ). The sort of future-oriented change and development required by this shift in institutional practices, programming, and structure will likely meet with significant resistance from comfortably entrenched (and often outdated) segments of traditional educational and training establishments. Additional external evaluation and monitoring is rarely welcome by workers in any context. We believe, however, that top-down processes from the innovative and competition-conscious administrative levels will be met by bottom-up demands from students and education consumers to support these institutional changes. And we contend that efforts such as labelizing 4C processes will serve to push educators and institutions towards more relevant offerings, oriented towards the future of work and helping build a more successful future for all.
In the end, the 4Cs framework seems to be a manageable, focused model for modernizing education, and one worthy of its growing prevalence in the educational and research marketplace for a number of reasons. These reasons include the complexity and cumbersome nature of larger alternative systems and the 4Cs’ persuasive presence at the core of a number of early and industry-driven frameworks. In addition, the 4Cs have benefitted from their subsequent promotion by organizations such as the OECD and the World Economic Forum, as well as some more direct support from recent empirical research. The promotion, teaching, and assessment of the 4Cs will require a complex social intervention and mobilization of educational resources—a major shift in pedagogy and institutional structures. Yet the same evolving digital technologies that have largely caused the need for these massive, rapid changes can also assist in the implementation of solutions ( van Laar et al. 2017 ). To the extent that future research also converges on such a model (that has already been found pedagogically useful and policy-friendly by so many individuals and organizations), the 4Cs framework has the potential to become a manageable core for 21st century skills and the future of education and work—one that stakeholders with various agendas can already begin building on for a better educational and economic future together.
This research received no external funding.
Conceptualization, B.T.-M. and T.L.; writing—original draft preparation, B.T.-M., A.C., M.M., J.-M.B., T.M., S.B.-B., S.E.H., F.V., M.A.-L., C.F., D.S., F.M.; writing—review and editing, B.T.-M., A.C., T.L., J.-M.B., C.F.; visualization, B.T.-M.; supervision, B.T.-M., T.L.; project administration, B.T.-M., T.L. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Data availability statement, conflicts of interest.
B.T.-M. and T.L. are unpaid academic co-founder and project collaborator for the International Institute for Competency Development, whose labelization frameworks (developed in cooperation with Afnor International and the LaPEA lab of Université Paris Cité and Université Gustave Eiffel) are used as examples in this review. S.E.H. and M.A.-L. are employees of AFNOR International. No funding was received to support this research or article, which reflects the views of the scientists and researchers and not their organizations or companies.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Search catalog, critical thinking and academic research: information.
Research involves gathering and interpreting information. To answer a question or understand the complexity of an issue, you have to seek relevant information, which helps you develop your own point of view.
It's important to remember, though, that information from outside sources should not stand in for your thinking. Sometimes, people think that gathering information and summarizing it in a paper is all there is to the research process. But finding information is just part of the process.
Research involves applying critical thinking to information, whether it comes from an encyclopedia entry, a journal article, a website, or a documentary. A researcher analyzes the material and develops a perspective on it. The goal is to think critically about the information, not simply repeat its ideas.
The purpose of your research and the questions you're trying to answer will determine what information is relevant and useful. If you're trying to understand public opinion on an issue, it might be worthwhile to look at news articles and blog entries. On the other hand, such sources may not be appropriate for a formal philosophical argument or a medical study.
Sources used in academic papers might include scholarly journals, books, research reports, government documents, films, comic books, magazines, newspapers, maps, statistics, letters, diaries, dictionaries, musical recordings, and more. It all depends on your purpose.
The information universe is very complex, so it's important to understand the differences among information sources. For instance, online information includes commercial websites, personal blogs, subscription databases, professional news sites, government resources, Wikipedia entries, Facebook profiles, Twitter feeds, YouTube videos, and much more. Different research projects require different types of sources. In many cases, you will need to look beyond the free web to find scholarly information in subscription library databases such as ProQuest Direct and EBSCO Academic Search Premier.
Understanding varying levels of complexity in information sources is also important. For example, a reference encyclopedia might provide useful background information on postmodernism, but it will not provide the level of sophistication and depth offered in an original work of postmodern theory or a scholarly article that applies that theory.
While background sources are useful and will help you understand more complex material, most professors expect you to explore in-depth, scholarly sources, most of which are not available on the free web. That's one reason why learning to use library resources is crucial.
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Humanities and Social Sciences Communications volume 10 , Article number: 16 ( 2023 ) Cite this article
18k Accesses
20 Citations
3 Altmetric
Metrics details
Collaborative problem-solving has been widely embraced in the classroom instruction of critical thinking, which is regarded as the core of curriculum reform based on key competencies in the field of education as well as a key competence for learners in the 21st century. However, the effectiveness of collaborative problem-solving in promoting students’ critical thinking remains uncertain. This current research presents the major findings of a meta-analysis of 36 pieces of the literature revealed in worldwide educational periodicals during the 21st century to identify the effectiveness of collaborative problem-solving in promoting students’ critical thinking and to determine, based on evidence, whether and to what extent collaborative problem solving can result in a rise or decrease in critical thinking. The findings show that (1) collaborative problem solving is an effective teaching approach to foster students’ critical thinking, with a significant overall effect size (ES = 0.82, z = 12.78, P < 0.01, 95% CI [0.69, 0.95]); (2) in respect to the dimensions of critical thinking, collaborative problem solving can significantly and successfully enhance students’ attitudinal tendencies (ES = 1.17, z = 7.62, P < 0.01, 95% CI[0.87, 1.47]); nevertheless, it falls short in terms of improving students’ cognitive skills, having only an upper-middle impact (ES = 0.70, z = 11.55, P < 0.01, 95% CI[0.58, 0.82]); and (3) the teaching type (chi 2 = 7.20, P < 0.05), intervention duration (chi 2 = 12.18, P < 0.01), subject area (chi 2 = 13.36, P < 0.05), group size (chi 2 = 8.77, P < 0.05), and learning scaffold (chi 2 = 9.03, P < 0.01) all have an impact on critical thinking, and they can be viewed as important moderating factors that affect how critical thinking develops. On the basis of these results, recommendations are made for further study and instruction to better support students’ critical thinking in the context of collaborative problem-solving.
Introduction.
Although critical thinking has a long history in research, the concept of critical thinking, which is regarded as an essential competence for learners in the 21st century, has recently attracted more attention from researchers and teaching practitioners (National Research Council, 2012 ). Critical thinking should be the core of curriculum reform based on key competencies in the field of education (Peng and Deng, 2017 ) because students with critical thinking can not only understand the meaning of knowledge but also effectively solve practical problems in real life even after knowledge is forgotten (Kek and Huijser, 2011 ). The definition of critical thinking is not universal (Ennis, 1989 ; Castle, 2009 ; Niu et al., 2013 ). In general, the definition of critical thinking is a self-aware and self-regulated thought process (Facione, 1990 ; Niu et al., 2013 ). It refers to the cognitive skills needed to interpret, analyze, synthesize, reason, and evaluate information as well as the attitudinal tendency to apply these abilities (Halpern, 2001 ). The view that critical thinking can be taught and learned through curriculum teaching has been widely supported by many researchers (e.g., Kuncel, 2011 ; Leng and Lu, 2020 ), leading to educators’ efforts to foster it among students. In the field of teaching practice, there are three types of courses for teaching critical thinking (Ennis, 1989 ). The first is an independent curriculum in which critical thinking is taught and cultivated without involving the knowledge of specific disciplines; the second is an integrated curriculum in which critical thinking is integrated into the teaching of other disciplines as a clear teaching goal; and the third is a mixed curriculum in which critical thinking is taught in parallel to the teaching of other disciplines for mixed teaching training. Furthermore, numerous measuring tools have been developed by researchers and educators to measure critical thinking in the context of teaching practice. These include standardized measurement tools, such as WGCTA, CCTST, CCTT, and CCTDI, which have been verified by repeated experiments and are considered effective and reliable by international scholars (Facione and Facione, 1992 ). In short, descriptions of critical thinking, including its two dimensions of attitudinal tendency and cognitive skills, different types of teaching courses, and standardized measurement tools provide a complex normative framework for understanding, teaching, and evaluating critical thinking.
Cultivating critical thinking in curriculum teaching can start with a problem, and one of the most popular critical thinking instructional approaches is problem-based learning (Liu et al., 2020 ). Duch et al. ( 2001 ) noted that problem-based learning in group collaboration is progressive active learning, which can improve students’ critical thinking and problem-solving skills. Collaborative problem-solving is the organic integration of collaborative learning and problem-based learning, which takes learners as the center of the learning process and uses problems with poor structure in real-world situations as the starting point for the learning process (Liang et al., 2017 ). Students learn the knowledge needed to solve problems in a collaborative group, reach a consensus on problems in the field, and form solutions through social cooperation methods, such as dialogue, interpretation, questioning, debate, negotiation, and reflection, thus promoting the development of learners’ domain knowledge and critical thinking (Cindy, 2004 ; Liang et al., 2017 ).
Collaborative problem-solving has been widely used in the teaching practice of critical thinking, and several studies have attempted to conduct a systematic review and meta-analysis of the empirical literature on critical thinking from various perspectives. However, little attention has been paid to the impact of collaborative problem-solving on critical thinking. Therefore, the best approach for developing and enhancing critical thinking throughout collaborative problem-solving is to examine how to implement critical thinking instruction; however, this issue is still unexplored, which means that many teachers are incapable of better instructing critical thinking (Leng and Lu, 2020 ; Niu et al., 2013 ). For example, Huber ( 2016 ) provided the meta-analysis findings of 71 publications on gaining critical thinking over various time frames in college with the aim of determining whether critical thinking was truly teachable. These authors found that learners significantly improve their critical thinking while in college and that critical thinking differs with factors such as teaching strategies, intervention duration, subject area, and teaching type. The usefulness of collaborative problem-solving in fostering students’ critical thinking, however, was not determined by this study, nor did it reveal whether there existed significant variations among the different elements. A meta-analysis of 31 pieces of educational literature was conducted by Liu et al. ( 2020 ) to assess the impact of problem-solving on college students’ critical thinking. These authors found that problem-solving could promote the development of critical thinking among college students and proposed establishing a reasonable group structure for problem-solving in a follow-up study to improve students’ critical thinking. Additionally, previous empirical studies have reached inconclusive and even contradictory conclusions about whether and to what extent collaborative problem-solving increases or decreases critical thinking levels. As an illustration, Yang et al. ( 2008 ) carried out an experiment on the integrated curriculum teaching of college students based on a web bulletin board with the goal of fostering participants’ critical thinking in the context of collaborative problem-solving. These authors’ research revealed that through sharing, debating, examining, and reflecting on various experiences and ideas, collaborative problem-solving can considerably enhance students’ critical thinking in real-life problem situations. In contrast, collaborative problem-solving had a positive impact on learners’ interaction and could improve learning interest and motivation but could not significantly improve students’ critical thinking when compared to traditional classroom teaching, according to research by Naber and Wyatt ( 2014 ) and Sendag and Odabasi ( 2009 ) on undergraduate and high school students, respectively.
The above studies show that there is inconsistency regarding the effectiveness of collaborative problem-solving in promoting students’ critical thinking. Therefore, it is essential to conduct a thorough and trustworthy review to detect and decide whether and to what degree collaborative problem-solving can result in a rise or decrease in critical thinking. Meta-analysis is a quantitative analysis approach that is utilized to examine quantitative data from various separate studies that are all focused on the same research topic. This approach characterizes the effectiveness of its impact by averaging the effect sizes of numerous qualitative studies in an effort to reduce the uncertainty brought on by independent research and produce more conclusive findings (Lipsey and Wilson, 2001 ).
This paper used a meta-analytic approach and carried out a meta-analysis to examine the effectiveness of collaborative problem-solving in promoting students’ critical thinking in order to make a contribution to both research and practice. The following research questions were addressed by this meta-analysis:
What is the overall effect size of collaborative problem-solving in promoting students’ critical thinking and its impact on the two dimensions of critical thinking (i.e., attitudinal tendency and cognitive skills)?
How are the disparities between the study conclusions impacted by various moderating variables if the impacts of various experimental designs in the included studies are heterogeneous?
This research followed the strict procedures (e.g., database searching, identification, screening, eligibility, merging, duplicate removal, and analysis of included studies) of Cooper’s ( 2010 ) proposed meta-analysis approach for examining quantitative data from various separate studies that are all focused on the same research topic. The relevant empirical research that appeared in worldwide educational periodicals within the 21st century was subjected to this meta-analysis using Rev-Man 5.4. The consistency of the data extracted separately by two researchers was tested using Cohen’s kappa coefficient, and a publication bias test and a heterogeneity test were run on the sample data to ascertain the quality of this meta-analysis.
There were three stages to the data collection process for this meta-analysis, as shown in Fig. 1 , which shows the number of articles included and eliminated during the selection process based on the statement and study eligibility criteria.
This flowchart shows the number of records identified, included and excluded in the article.
First, the databases used to systematically search for relevant articles were the journal papers of the Web of Science Core Collection and the Chinese Core source journal, as well as the Chinese Social Science Citation Index (CSSCI) source journal papers included in CNKI. These databases were selected because they are credible platforms that are sources of scholarly and peer-reviewed information with advanced search tools and contain literature relevant to the subject of our topic from reliable researchers and experts. The search string with the Boolean operator used in the Web of Science was “TS = (((“critical thinking” or “ct” and “pretest” or “posttest”) or (“critical thinking” or “ct” and “control group” or “quasi experiment” or “experiment”)) and (“collaboration” or “collaborative learning” or “CSCL”) and (“problem solving” or “problem-based learning” or “PBL”))”. The research area was “Education Educational Research”, and the search period was “January 1, 2000, to December 30, 2021”. A total of 412 papers were obtained. The search string with the Boolean operator used in the CNKI was “SU = (‘critical thinking’*‘collaboration’ + ‘critical thinking’*‘collaborative learning’ + ‘critical thinking’*‘CSCL’ + ‘critical thinking’*‘problem solving’ + ‘critical thinking’*‘problem-based learning’ + ‘critical thinking’*‘PBL’ + ‘critical thinking’*‘problem oriented’) AND FT = (‘experiment’ + ‘quasi experiment’ + ‘pretest’ + ‘posttest’ + ‘empirical study’)” (translated into Chinese when searching). A total of 56 studies were found throughout the search period of “January 2000 to December 2021”. From the databases, all duplicates and retractions were eliminated before exporting the references into Endnote, a program for managing bibliographic references. In all, 466 studies were found.
Second, the studies that matched the inclusion and exclusion criteria for the meta-analysis were chosen by two researchers after they had reviewed the abstracts and titles of the gathered articles, yielding a total of 126 studies.
Third, two researchers thoroughly reviewed each included article’s whole text in accordance with the inclusion and exclusion criteria. Meanwhile, a snowball search was performed using the references and citations of the included articles to ensure complete coverage of the articles. Ultimately, 36 articles were kept.
Two researchers worked together to carry out this entire process, and a consensus rate of almost 94.7% was reached after discussion and negotiation to clarify any emerging differences.
Since not all the retrieved studies matched the criteria for this meta-analysis, eligibility criteria for both inclusion and exclusion were developed as follows:
The publication language of the included studies was limited to English and Chinese, and the full text could be obtained. Articles that did not meet the publication language and articles not published between 2000 and 2021 were excluded.
The research design of the included studies must be empirical and quantitative studies that can assess the effect of collaborative problem-solving on the development of critical thinking. Articles that could not identify the causal mechanisms by which collaborative problem-solving affects critical thinking, such as review articles and theoretical articles, were excluded.
The research method of the included studies must feature a randomized control experiment or a quasi-experiment, or a natural experiment, which have a higher degree of internal validity with strong experimental designs and can all plausibly provide evidence that critical thinking and collaborative problem-solving are causally related. Articles with non-experimental research methods, such as purely correlational or observational studies, were excluded.
The participants of the included studies were only students in school, including K-12 students and college students. Articles in which the participants were non-school students, such as social workers or adult learners, were excluded.
The research results of the included studies must mention definite signs that may be utilized to gauge critical thinking’s impact (e.g., sample size, mean value, or standard deviation). Articles that lacked specific measurement indicators for critical thinking and could not calculate the effect size were excluded.
In order to perform a meta-analysis, it is necessary to collect the most important information from the articles, codify that information’s properties, and convert descriptive data into quantitative data. Therefore, this study designed a data coding template (see Table 1 ). Ultimately, 16 coding fields were retained.
The designed data-coding template consisted of three pieces of information. Basic information about the papers was included in the descriptive information: the publishing year, author, serial number, and title of the paper.
The variable information for the experimental design had three variables: the independent variable (instruction method), the dependent variable (critical thinking), and the moderating variable (learning stage, teaching type, intervention duration, learning scaffold, group size, measuring tool, and subject area). Depending on the topic of this study, the intervention strategy, as the independent variable, was coded into collaborative and non-collaborative problem-solving. The dependent variable, critical thinking, was coded as a cognitive skill and an attitudinal tendency. And seven moderating variables were created by grouping and combining the experimental design variables discovered within the 36 studies (see Table 1 ), where learning stages were encoded as higher education, high school, middle school, and primary school or lower; teaching types were encoded as mixed courses, integrated courses, and independent courses; intervention durations were encoded as 0–1 weeks, 1–4 weeks, 4–12 weeks, and more than 12 weeks; group sizes were encoded as 2–3 persons, 4–6 persons, 7–10 persons, and more than 10 persons; learning scaffolds were encoded as teacher-supported learning scaffold, technique-supported learning scaffold, and resource-supported learning scaffold; measuring tools were encoded as standardized measurement tools (e.g., WGCTA, CCTT, CCTST, and CCTDI) and self-adapting measurement tools (e.g., modified or made by researchers); and subject areas were encoded according to the specific subjects used in the 36 included studies.
The data information contained three metrics for measuring critical thinking: sample size, average value, and standard deviation. It is vital to remember that studies with various experimental designs frequently adopt various formulas to determine the effect size. And this paper used Morris’ proposed standardized mean difference (SMD) calculation formula ( 2008 , p. 369; see Supplementary Table S3 ).
According to the data coding template (see Table 1 ), the 36 papers’ information was retrieved by two researchers, who then entered them into Excel (see Supplementary Table S1 ). The results of each study were extracted separately in the data extraction procedure if an article contained numerous studies on critical thinking, or if a study assessed different critical thinking dimensions. For instance, Tiwari et al. ( 2010 ) used four time points, which were viewed as numerous different studies, to examine the outcomes of critical thinking, and Chen ( 2013 ) included the two outcome variables of attitudinal tendency and cognitive skills, which were regarded as two studies. After discussion and negotiation during data extraction, the two researchers’ consistency test coefficients were roughly 93.27%. Supplementary Table S2 details the key characteristics of the 36 included articles with 79 effect quantities, including descriptive information (e.g., the publishing year, author, serial number, and title of the paper), variable information (e.g., independent variables, dependent variables, and moderating variables), and data information (e.g., mean values, standard deviations, and sample size). Following that, testing for publication bias and heterogeneity was done on the sample data using the Rev-Man 5.4 software, and then the test results were used to conduct a meta-analysis.
When the sample of studies included in a meta-analysis does not accurately reflect the general status of research on the relevant subject, publication bias is said to be exhibited in this research. The reliability and accuracy of the meta-analysis may be impacted by publication bias. Due to this, the meta-analysis needs to check the sample data for publication bias (Stewart et al., 2006 ). A popular method to check for publication bias is the funnel plot; and it is unlikely that there will be publishing bias when the data are equally dispersed on either side of the average effect size and targeted within the higher region. The data are equally dispersed within the higher portion of the efficient zone, consistent with the funnel plot connected with this analysis (see Fig. 2 ), indicating that publication bias is unlikely in this situation.
This funnel plot shows the result of publication bias of 79 effect quantities across 36 studies.
To select the appropriate effect models for the meta-analysis, one might use the results of a heterogeneity test on the data effect sizes. In a meta-analysis, it is common practice to gauge the degree of data heterogeneity using the I 2 value, and I 2 ≥ 50% is typically understood to denote medium-high heterogeneity, which calls for the adoption of a random effect model; if not, a fixed effect model ought to be applied (Lipsey and Wilson, 2001 ). The findings of the heterogeneity test in this paper (see Table 2 ) revealed that I 2 was 86% and displayed significant heterogeneity ( P < 0.01). To ensure accuracy and reliability, the overall effect size ought to be calculated utilizing the random effect model.
This meta-analysis utilized a random effect model to examine 79 effect quantities from 36 studies after eliminating heterogeneity. In accordance with Cohen’s criterion (Cohen, 1992 ), it is abundantly clear from the analysis results, which are shown in the forest plot of the overall effect (see Fig. 3 ), that the cumulative impact size of cooperative problem-solving is 0.82, which is statistically significant ( z = 12.78, P < 0.01, 95% CI [0.69, 0.95]), and can encourage learners to practice critical thinking.
This forest plot shows the analysis result of the overall effect size across 36 studies.
In addition, this study examined two distinct dimensions of critical thinking to better understand the precise contributions that collaborative problem-solving makes to the growth of critical thinking. The findings (see Table 3 ) indicate that collaborative problem-solving improves cognitive skills (ES = 0.70) and attitudinal tendency (ES = 1.17), with significant intergroup differences (chi 2 = 7.95, P < 0.01). Although collaborative problem-solving improves both dimensions of critical thinking, it is essential to point out that the improvements in students’ attitudinal tendency are much more pronounced and have a significant comprehensive effect (ES = 1.17, z = 7.62, P < 0.01, 95% CI [0.87, 1.47]), whereas gains in learners’ cognitive skill are slightly improved and are just above average. (ES = 0.70, z = 11.55, P < 0.01, 95% CI [0.58, 0.82]).
The whole forest plot’s 79 effect quantities underwent a two-tailed test, which revealed significant heterogeneity ( I 2 = 86%, z = 12.78, P < 0.01), indicating differences between various effect sizes that may have been influenced by moderating factors other than sampling error. Therefore, exploring possible moderating factors that might produce considerable heterogeneity was done using subgroup analysis, such as the learning stage, learning scaffold, teaching type, group size, duration of the intervention, measuring tool, and the subject area included in the 36 experimental designs, in order to further explore the key factors that influence critical thinking. The findings (see Table 4 ) indicate that various moderating factors have advantageous effects on critical thinking. In this situation, the subject area (chi 2 = 13.36, P < 0.05), group size (chi 2 = 8.77, P < 0.05), intervention duration (chi 2 = 12.18, P < 0.01), learning scaffold (chi 2 = 9.03, P < 0.01), and teaching type (chi 2 = 7.20, P < 0.05) are all significant moderators that can be applied to support the cultivation of critical thinking. However, since the learning stage and the measuring tools did not significantly differ among intergroup (chi 2 = 3.15, P = 0.21 > 0.05, and chi 2 = 0.08, P = 0.78 > 0.05), we are unable to explain why these two factors are crucial in supporting the cultivation of critical thinking in the context of collaborative problem-solving. These are the precise outcomes, as follows:
Various learning stages influenced critical thinking positively, without significant intergroup differences (chi 2 = 3.15, P = 0.21 > 0.05). High school was first on the list of effect sizes (ES = 1.36, P < 0.01), then higher education (ES = 0.78, P < 0.01), and middle school (ES = 0.73, P < 0.01). These results show that, despite the learning stage’s beneficial influence on cultivating learners’ critical thinking, we are unable to explain why it is essential for cultivating critical thinking in the context of collaborative problem-solving.
Different teaching types had varying degrees of positive impact on critical thinking, with significant intergroup differences (chi 2 = 7.20, P < 0.05). The effect size was ranked as follows: mixed courses (ES = 1.34, P < 0.01), integrated courses (ES = 0.81, P < 0.01), and independent courses (ES = 0.27, P < 0.01). These results indicate that the most effective approach to cultivate critical thinking utilizing collaborative problem solving is through the teaching type of mixed courses.
Various intervention durations significantly improved critical thinking, and there were significant intergroup differences (chi 2 = 12.18, P < 0.01). The effect sizes related to this variable showed a tendency to increase with longer intervention durations. The improvement in critical thinking reached a significant level (ES = 0.85, P < 0.01) after more than 12 weeks of training. These findings indicate that the intervention duration and critical thinking’s impact are positively correlated, with a longer intervention duration having a greater effect.
Different learning scaffolds influenced critical thinking positively, with significant intergroup differences (chi 2 = 9.03, P < 0.01). The resource-supported learning scaffold (ES = 0.69, P < 0.01) acquired a medium-to-higher level of impact, the technique-supported learning scaffold (ES = 0.63, P < 0.01) also attained a medium-to-higher level of impact, and the teacher-supported learning scaffold (ES = 0.92, P < 0.01) displayed a high level of significant impact. These results show that the learning scaffold with teacher support has the greatest impact on cultivating critical thinking.
Various group sizes influenced critical thinking positively, and the intergroup differences were statistically significant (chi 2 = 8.77, P < 0.05). Critical thinking showed a general declining trend with increasing group size. The overall effect size of 2–3 people in this situation was the biggest (ES = 0.99, P < 0.01), and when the group size was greater than 7 people, the improvement in critical thinking was at the lower-middle level (ES < 0.5, P < 0.01). These results show that the impact on critical thinking is positively connected with group size, and as group size grows, so does the overall impact.
Various measuring tools influenced critical thinking positively, with significant intergroup differences (chi 2 = 0.08, P = 0.78 > 0.05). In this situation, the self-adapting measurement tools obtained an upper-medium level of effect (ES = 0.78), whereas the complete effect size of the standardized measurement tools was the largest, achieving a significant level of effect (ES = 0.84, P < 0.01). These results show that, despite the beneficial influence of the measuring tool on cultivating critical thinking, we are unable to explain why it is crucial in fostering the growth of critical thinking by utilizing the approach of collaborative problem-solving.
Different subject areas had a greater impact on critical thinking, and the intergroup differences were statistically significant (chi 2 = 13.36, P < 0.05). Mathematics had the greatest overall impact, achieving a significant level of effect (ES = 1.68, P < 0.01), followed by science (ES = 1.25, P < 0.01) and medical science (ES = 0.87, P < 0.01), both of which also achieved a significant level of effect. Programming technology was the least effective (ES = 0.39, P < 0.01), only having a medium-low degree of effect compared to education (ES = 0.72, P < 0.01) and other fields (such as language, art, and social sciences) (ES = 0.58, P < 0.01). These results suggest that scientific fields (e.g., mathematics, science) may be the most effective subject areas for cultivating critical thinking utilizing the approach of collaborative problem-solving.
According to this meta-analysis, using collaborative problem-solving as an intervention strategy in critical thinking teaching has a considerable amount of impact on cultivating learners’ critical thinking as a whole and has a favorable promotional effect on the two dimensions of critical thinking. According to certain studies, collaborative problem solving, the most frequently used critical thinking teaching strategy in curriculum instruction can considerably enhance students’ critical thinking (e.g., Liang et al., 2017 ; Liu et al., 2020 ; Cindy, 2004 ). This meta-analysis provides convergent data support for the above research views. Thus, the findings of this meta-analysis not only effectively address the first research query regarding the overall effect of cultivating critical thinking and its impact on the two dimensions of critical thinking (i.e., attitudinal tendency and cognitive skills) utilizing the approach of collaborative problem-solving, but also enhance our confidence in cultivating critical thinking by using collaborative problem-solving intervention approach in the context of classroom teaching.
Furthermore, the associated improvements in attitudinal tendency are much stronger, but the corresponding improvements in cognitive skill are only marginally better. According to certain studies, cognitive skill differs from the attitudinal tendency in classroom instruction; the cultivation and development of the former as a key ability is a process of gradual accumulation, while the latter as an attitude is affected by the context of the teaching situation (e.g., a novel and exciting teaching approach, challenging and rewarding tasks) (Halpern, 2001 ; Wei and Hong, 2022 ). Collaborative problem-solving as a teaching approach is exciting and interesting, as well as rewarding and challenging; because it takes the learners as the focus and examines problems with poor structure in real situations, and it can inspire students to fully realize their potential for problem-solving, which will significantly improve their attitudinal tendency toward solving problems (Liu et al., 2020 ). Similar to how collaborative problem-solving influences attitudinal tendency, attitudinal tendency impacts cognitive skill when attempting to solve a problem (Liu et al., 2020 ; Zhang et al., 2022 ), and stronger attitudinal tendencies are associated with improved learning achievement and cognitive ability in students (Sison, 2008 ; Zhang et al., 2022 ). It can be seen that the two specific dimensions of critical thinking as well as critical thinking as a whole are affected by collaborative problem-solving, and this study illuminates the nuanced links between cognitive skills and attitudinal tendencies with regard to these two dimensions of critical thinking. To fully develop students’ capacity for critical thinking, future empirical research should pay closer attention to cognitive skills.
In order to further explore the key factors that influence critical thinking, exploring possible moderating effects that might produce considerable heterogeneity was done using subgroup analysis. The findings show that the moderating factors, such as the teaching type, learning stage, group size, learning scaffold, duration of the intervention, measuring tool, and the subject area included in the 36 experimental designs, could all support the cultivation of collaborative problem-solving in critical thinking. Among them, the effect size differences between the learning stage and measuring tool are not significant, which does not explain why these two factors are crucial in supporting the cultivation of critical thinking utilizing the approach of collaborative problem-solving.
In terms of the learning stage, various learning stages influenced critical thinking positively without significant intergroup differences, indicating that we are unable to explain why it is crucial in fostering the growth of critical thinking.
Although high education accounts for 70.89% of all empirical studies performed by researchers, high school may be the appropriate learning stage to foster students’ critical thinking by utilizing the approach of collaborative problem-solving since it has the largest overall effect size. This phenomenon may be related to student’s cognitive development, which needs to be further studied in follow-up research.
With regard to teaching type, mixed course teaching may be the best teaching method to cultivate students’ critical thinking. Relevant studies have shown that in the actual teaching process if students are trained in thinking methods alone, the methods they learn are isolated and divorced from subject knowledge, which is not conducive to their transfer of thinking methods; therefore, if students’ thinking is trained only in subject teaching without systematic method training, it is challenging to apply to real-world circumstances (Ruggiero, 2012 ; Hu and Liu, 2015 ). Teaching critical thinking as mixed course teaching in parallel to other subject teachings can achieve the best effect on learners’ critical thinking, and explicit critical thinking instruction is more effective than less explicit critical thinking instruction (Bensley and Spero, 2014 ).
In terms of the intervention duration, with longer intervention times, the overall effect size shows an upward tendency. Thus, the intervention duration and critical thinking’s impact are positively correlated. Critical thinking, as a key competency for students in the 21st century, is difficult to get a meaningful improvement in a brief intervention duration. Instead, it could be developed over a lengthy period of time through consistent teaching and the progressive accumulation of knowledge (Halpern, 2001 ; Hu and Liu, 2015 ). Therefore, future empirical studies ought to take these restrictions into account throughout a longer period of critical thinking instruction.
With regard to group size, a group size of 2–3 persons has the highest effect size, and the comprehensive effect size decreases with increasing group size in general. This outcome is in line with some research findings; as an example, a group composed of two to four members is most appropriate for collaborative learning (Schellens and Valcke, 2006 ). However, the meta-analysis results also indicate that once the group size exceeds 7 people, small groups cannot produce better interaction and performance than large groups. This may be because the learning scaffolds of technique support, resource support, and teacher support improve the frequency and effectiveness of interaction among group members, and a collaborative group with more members may increase the diversity of views, which is helpful to cultivate critical thinking utilizing the approach of collaborative problem-solving.
With regard to the learning scaffold, the three different kinds of learning scaffolds can all enhance critical thinking. Among them, the teacher-supported learning scaffold has the largest overall effect size, demonstrating the interdependence of effective learning scaffolds and collaborative problem-solving. This outcome is in line with some research findings; as an example, a successful strategy is to encourage learners to collaborate, come up with solutions, and develop critical thinking skills by using learning scaffolds (Reiser, 2004 ; Xu et al., 2022 ); learning scaffolds can lower task complexity and unpleasant feelings while also enticing students to engage in learning activities (Wood et al., 2006 ); learning scaffolds are designed to assist students in using learning approaches more successfully to adapt the collaborative problem-solving process, and the teacher-supported learning scaffolds have the greatest influence on critical thinking in this process because they are more targeted, informative, and timely (Xu et al., 2022 ).
With respect to the measuring tool, despite the fact that standardized measurement tools (such as the WGCTA, CCTT, and CCTST) have been acknowledged as trustworthy and effective by worldwide experts, only 54.43% of the research included in this meta-analysis adopted them for assessment, and the results indicated no intergroup differences. These results suggest that not all teaching circumstances are appropriate for measuring critical thinking using standardized measurement tools. “The measuring tools for measuring thinking ability have limits in assessing learners in educational situations and should be adapted appropriately to accurately assess the changes in learners’ critical thinking.”, according to Simpson and Courtney ( 2002 , p. 91). As a result, in order to more fully and precisely gauge how learners’ critical thinking has evolved, we must properly modify standardized measuring tools based on collaborative problem-solving learning contexts.
With regard to the subject area, the comprehensive effect size of science departments (e.g., mathematics, science, medical science) is larger than that of language arts and social sciences. Some recent international education reforms have noted that critical thinking is a basic part of scientific literacy. Students with scientific literacy can prove the rationality of their judgment according to accurate evidence and reasonable standards when they face challenges or poorly structured problems (Kyndt et al., 2013 ), which makes critical thinking crucial for developing scientific understanding and applying this understanding to practical problem solving for problems related to science, technology, and society (Yore et al., 2007 ).
Other than those stated in the discussion above, the following suggestions are offered for critical thinking instruction utilizing the approach of collaborative problem-solving.
First, teachers should put a special emphasis on the two core elements, which are collaboration and problem-solving, to design real problems based on collaborative situations. This meta-analysis provides evidence to support the view that collaborative problem-solving has a strong synergistic effect on promoting students’ critical thinking. Asking questions about real situations and allowing learners to take part in critical discussions on real problems during class instruction are key ways to teach critical thinking rather than simply reading speculative articles without practice (Mulnix, 2012 ). Furthermore, the improvement of students’ critical thinking is realized through cognitive conflict with other learners in the problem situation (Yang et al., 2008 ). Consequently, it is essential for teachers to put a special emphasis on the two core elements, which are collaboration and problem-solving, and design real problems and encourage students to discuss, negotiate, and argue based on collaborative problem-solving situations.
Second, teachers should design and implement mixed courses to cultivate learners’ critical thinking, utilizing the approach of collaborative problem-solving. Critical thinking can be taught through curriculum instruction (Kuncel, 2011 ; Leng and Lu, 2020 ), with the goal of cultivating learners’ critical thinking for flexible transfer and application in real problem-solving situations. This meta-analysis shows that mixed course teaching has a highly substantial impact on the cultivation and promotion of learners’ critical thinking. Therefore, teachers should design and implement mixed course teaching with real collaborative problem-solving situations in combination with the knowledge content of specific disciplines in conventional teaching, teach methods and strategies of critical thinking based on poorly structured problems to help students master critical thinking, and provide practical activities in which students can interact with each other to develop knowledge construction and critical thinking utilizing the approach of collaborative problem-solving.
Third, teachers should be more trained in critical thinking, particularly preservice teachers, and they also should be conscious of the ways in which teachers’ support for learning scaffolds can promote critical thinking. The learning scaffold supported by teachers had the greatest impact on learners’ critical thinking, in addition to being more directive, targeted, and timely (Wood et al., 2006 ). Critical thinking can only be effectively taught when teachers recognize the significance of critical thinking for students’ growth and use the proper approaches while designing instructional activities (Forawi, 2016 ). Therefore, with the intention of enabling teachers to create learning scaffolds to cultivate learners’ critical thinking utilizing the approach of collaborative problem solving, it is essential to concentrate on the teacher-supported learning scaffolds and enhance the instruction for teaching critical thinking to teachers, especially preservice teachers.
There are certain limitations in this meta-analysis, but future research can correct them. First, the search languages were restricted to English and Chinese, so it is possible that pertinent studies that were written in other languages were overlooked, resulting in an inadequate number of articles for review. Second, these data provided by the included studies are partially missing, such as whether teachers were trained in the theory and practice of critical thinking, the average age and gender of learners, and the differences in critical thinking among learners of various ages and genders. Third, as is typical for review articles, more studies were released while this meta-analysis was being done; therefore, it had a time limit. With the development of relevant research, future studies focusing on these issues are highly relevant and needed.
The subject of the magnitude of collaborative problem-solving’s impact on fostering students’ critical thinking, which received scant attention from other studies, was successfully addressed by this study. The question of the effectiveness of collaborative problem-solving in promoting students’ critical thinking was addressed in this study, which addressed a topic that had gotten little attention in earlier research. The following conclusions can be made:
Regarding the results obtained, collaborative problem solving is an effective teaching approach to foster learners’ critical thinking, with a significant overall effect size (ES = 0.82, z = 12.78, P < 0.01, 95% CI [0.69, 0.95]). With respect to the dimensions of critical thinking, collaborative problem-solving can significantly and effectively improve students’ attitudinal tendency, and the comprehensive effect is significant (ES = 1.17, z = 7.62, P < 0.01, 95% CI [0.87, 1.47]); nevertheless, it falls short in terms of improving students’ cognitive skills, having only an upper-middle impact (ES = 0.70, z = 11.55, P < 0.01, 95% CI [0.58, 0.82]).
As demonstrated by both the results and the discussion, there are varying degrees of beneficial effects on students’ critical thinking from all seven moderating factors, which were found across 36 studies. In this context, the teaching type (chi 2 = 7.20, P < 0.05), intervention duration (chi 2 = 12.18, P < 0.01), subject area (chi 2 = 13.36, P < 0.05), group size (chi 2 = 8.77, P < 0.05), and learning scaffold (chi 2 = 9.03, P < 0.01) all have a positive impact on critical thinking, and they can be viewed as important moderating factors that affect how critical thinking develops. Since the learning stage (chi 2 = 3.15, P = 0.21 > 0.05) and measuring tools (chi 2 = 0.08, P = 0.78 > 0.05) did not demonstrate any significant intergroup differences, we are unable to explain why these two factors are crucial in supporting the cultivation of critical thinking in the context of collaborative problem-solving.
All data generated or analyzed during this study are included within the article and its supplementary information files, and the supplementary information files are available in the Dataverse repository: https://doi.org/10.7910/DVN/IPFJO6 .
Bensley DA, Spero RA (2014) Improving critical thinking skills and meta-cognitive monitoring through direct infusion. Think Skills Creat 12:55–68. https://doi.org/10.1016/j.tsc.2014.02.001
Article Google Scholar
Castle A (2009) Defining and assessing critical thinking skills for student radiographers. Radiography 15(1):70–76. https://doi.org/10.1016/j.radi.2007.10.007
Chen XD (2013) An empirical study on the influence of PBL teaching model on critical thinking ability of non-English majors. J PLA Foreign Lang College 36 (04):68–72
Google Scholar
Cohen A (1992) Antecedents of organizational commitment across occupational groups: a meta-analysis. J Organ Behav. https://doi.org/10.1002/job.4030130602
Cooper H (2010) Research synthesis and meta-analysis: a step-by-step approach, 4th edn. Sage, London, England
Cindy HS (2004) Problem-based learning: what and how do students learn? Educ Psychol Rev 51(1):31–39
Duch BJ, Gron SD, Allen DE (2001) The power of problem-based learning: a practical “how to” for teaching undergraduate courses in any discipline. Stylus Educ Sci 2:190–198
Ennis RH (1989) Critical thinking and subject specificity: clarification and needed research. Educ Res 18(3):4–10. https://doi.org/10.3102/0013189x018003004
Facione PA (1990) Critical thinking: a statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations. Eric document reproduction service. https://eric.ed.gov/?id=ed315423
Facione PA, Facione NC (1992) The California Critical Thinking Dispositions Inventory (CCTDI) and the CCTDI test manual. California Academic Press, Millbrae, CA
Forawi SA (2016) Standard-based science education and critical thinking. Think Skills Creat 20:52–62. https://doi.org/10.1016/j.tsc.2016.02.005
Halpern DF (2001) Assessing the effectiveness of critical thinking instruction. J Gen Educ 50(4):270–286. https://doi.org/10.2307/27797889
Hu WP, Liu J (2015) Cultivation of pupils’ thinking ability: a five-year follow-up study. Psychol Behav Res 13(05):648–654. https://doi.org/10.3969/j.issn.1672-0628.2015.05.010
Huber K (2016) Does college teach critical thinking? A meta-analysis. Rev Educ Res 86(2):431–468. https://doi.org/10.3102/0034654315605917
Kek MYCA, Huijser H (2011) The power of problem-based learning in developing critical thinking skills: preparing students for tomorrow’s digital futures in today’s classrooms. High Educ Res Dev 30(3):329–341. https://doi.org/10.1080/07294360.2010.501074
Kuncel NR (2011) Measurement and meaning of critical thinking (Research report for the NRC 21st Century Skills Workshop). National Research Council, Washington, DC
Kyndt E, Raes E, Lismont B, Timmers F, Cascallar E, Dochy F (2013) A meta-analysis of the effects of face-to-face cooperative learning. Do recent studies falsify or verify earlier findings? Educ Res Rev 10(2):133–149. https://doi.org/10.1016/j.edurev.2013.02.002
Leng J, Lu XX (2020) Is critical thinking really teachable?—A meta-analysis based on 79 experimental or quasi experimental studies. Open Educ Res 26(06):110–118. https://doi.org/10.13966/j.cnki.kfjyyj.2020.06.011
Liang YZ, Zhu K, Zhao CL (2017) An empirical study on the depth of interaction promoted by collaborative problem solving learning activities. J E-educ Res 38(10):87–92. https://doi.org/10.13811/j.cnki.eer.2017.10.014
Lipsey M, Wilson D (2001) Practical meta-analysis. International Educational and Professional, London, pp. 92–160
Liu Z, Wu W, Jiang Q (2020) A study on the influence of problem based learning on college students’ critical thinking-based on a meta-analysis of 31 studies. Explor High Educ 03:43–49
Morris SB (2008) Estimating effect sizes from pretest-posttest-control group designs. Organ Res Methods 11(2):364–386. https://doi.org/10.1177/1094428106291059
Article ADS Google Scholar
Mulnix JW (2012) Thinking critically about critical thinking. Educ Philos Theory 44(5):464–479. https://doi.org/10.1111/j.1469-5812.2010.00673.x
Naber J, Wyatt TH (2014) The effect of reflective writing interventions on the critical thinking skills and dispositions of baccalaureate nursing students. Nurse Educ Today 34(1):67–72. https://doi.org/10.1016/j.nedt.2013.04.002
National Research Council (2012) Education for life and work: developing transferable knowledge and skills in the 21st century. The National Academies Press, Washington, DC
Niu L, Behar HLS, Garvan CW (2013) Do instructional interventions influence college students’ critical thinking skills? A meta-analysis. Educ Res Rev 9(12):114–128. https://doi.org/10.1016/j.edurev.2012.12.002
Peng ZM, Deng L (2017) Towards the core of education reform: cultivating critical thinking skills as the core of skills in the 21st century. Res Educ Dev 24:57–63. https://doi.org/10.14121/j.cnki.1008-3855.2017.24.011
Reiser BJ (2004) Scaffolding complex learning: the mechanisms of structuring and problematizing student work. J Learn Sci 13(3):273–304. https://doi.org/10.1207/s15327809jls1303_2
Ruggiero VR (2012) The art of thinking: a guide to critical and creative thought, 4th edn. Harper Collins College Publishers, New York
Schellens T, Valcke M (2006) Fostering knowledge construction in university students through asynchronous discussion groups. Comput Educ 46(4):349–370. https://doi.org/10.1016/j.compedu.2004.07.010
Sendag S, Odabasi HF (2009) Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills. Comput Educ 53(1):132–141. https://doi.org/10.1016/j.compedu.2009.01.008
Sison R (2008) Investigating Pair Programming in a Software Engineering Course in an Asian Setting. 2008 15th Asia-Pacific Software Engineering Conference, pp. 325–331. https://doi.org/10.1109/APSEC.2008.61
Simpson E, Courtney M (2002) Critical thinking in nursing education: literature review. Mary Courtney 8(2):89–98
Stewart L, Tierney J, Burdett S (2006) Do systematic reviews based on individual patient data offer a means of circumventing biases associated with trial publications? Publication bias in meta-analysis. John Wiley and Sons Inc, New York, pp. 261–286
Tiwari A, Lai P, So M, Yuen K (2010) A comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking. Med Educ 40(6):547–554. https://doi.org/10.1111/j.1365-2929.2006.02481.x
Wood D, Bruner JS, Ross G (2006) The role of tutoring in problem solving. J Child Psychol Psychiatry 17(2):89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
Wei T, Hong S (2022) The meaning and realization of teachable critical thinking. Educ Theory Practice 10:51–57
Xu EW, Wang W, Wang QX (2022) A meta-analysis of the effectiveness of programming teaching in promoting K-12 students’ computational thinking. Educ Inf Technol. https://doi.org/10.1007/s10639-022-11445-2
Yang YC, Newby T, Bill R (2008) Facilitating interactions through structured web-based bulletin boards: a quasi-experimental study on promoting learners’ critical thinking skills. Comput Educ 50(4):1572–1585. https://doi.org/10.1016/j.compedu.2007.04.006
Yore LD, Pimm D, Tuan HL (2007) The literacy component of mathematical and scientific literacy. Int J Sci Math Educ 5(4):559–589. https://doi.org/10.1007/s10763-007-9089-4
Zhang T, Zhang S, Gao QQ, Wang JH (2022) Research on the development of learners’ critical thinking in online peer review. Audio Visual Educ Res 6:53–60. https://doi.org/10.13811/j.cnki.eer.2022.06.08
Download references
This research was supported by the graduate scientific research and innovation project of Xinjiang Uygur Autonomous Region named “Research on in-depth learning of high school information technology courses for the cultivation of computing thinking” (No. XJ2022G190) and the independent innovation fund project for doctoral students of the College of Educational Science of Xinjiang Normal University named “Research on project-based teaching of high school information technology courses from the perspective of discipline core literacy” (No. XJNUJKYA2003).
Authors and affiliations.
College of Educational Science, Xinjiang Normal University, 830017, Urumqi, Xinjiang, China
Enwei Xu, Wei Wang & Qingxia Wang
You can also search for this author in PubMed Google Scholar
Correspondence to Enwei Xu or Wei Wang .
Competing interests.
The authors declare no competing interests.
This article does not contain any studies with human participants performed by any of the authors.
Additional information.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary tables, rights and permissions.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .
Reprints and permissions
Cite this article.
Xu, E., Wang, W. & Wang, Q. The effectiveness of collaborative problem solving in promoting students’ critical thinking: A meta-analysis based on empirical literature. Humanit Soc Sci Commun 10 , 16 (2023). https://doi.org/10.1057/s41599-023-01508-1
Download citation
Received : 07 August 2022
Accepted : 04 January 2023
Published : 11 January 2023
DOI : https://doi.org/10.1057/s41599-023-01508-1
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Impacts of online collaborative learning on students’ intercultural communication apprehension and intercultural communicative competence.
Education and Information Technologies (2024)
The impacts of computer-supported collaborative learning on students’ critical thinking: a meta-analysis.
International Journal of Environmental Science and Technology (2024)
IMAGES
VIDEO
COMMENTS
The importance of critical thinking in academics cannot be overstated. It equips students with the skills to discern credible sources from unreliable ones, develop well-informed arguments, and approach problems with a solution-oriented mindset.
Critical thinking is important for making judgments about sources of information and forming your own arguments. It emphasizes a rational, objective, and self-aware approach that can help you to identify credible sources and strengthen your conclusions.
When it comes to university contexts, critical thinking is defined in terms of abilities or skills such as selection, evaluation, analysis, reflection, questioning, inference and judgement (Tapper 2004).
Critical Thinking in Academic Research - 2nd Edition provides examples and easy-to-understand explanations to equip students with the skills to develop research questions, evaluate and choose the right sources, search for information, and understand arguments.
Critical thinking is widely recognized as a core competency and as a precursor to research. Employers value it as a requirement for every position they post, and every survey of potential employers for graduates in local markets rate the skill as their number one concern.
This guide explains the fundamental role of critical thinking in the academic research process.
By developing critical thinking skills, students develop the reasoning tools that can reorient their beliefs and values. Therefore, critical thinking can result in a transformative experience and, in turn, transformative learning.
This article addresses some of these challenges and related issues for the future of education and work, by focusing on so-called “21st Century Skills” and key “soft skills” known as the “4Cs” (creativity, critical thinking, communication, and collaboration), more particularly.
This guide explains the fundamental role of critical thinking in the academic research process. Intro. Purpose. Questions. Information. Concepts. Point of View. Inferences. Assumptions. Implications. Gather the Information. Research involves gathering and interpreting information.
The findings show that (1) collaborative problem solving is an effective teaching approach to foster students’ critical thinking, with a significant overall effect size (ES = 0.82, z = 12.78,...