Did students really care about creative thinking assessment in PISA 2022?

Abstract

The Programme for International Student Assessment (PISA) 2022 assessed creative thinking (CT) skills for the first time. While this assessment offers a unique snapshot of the creative abilities of 15-year-olds worldwide, it also raises questions about the influence of the low-stakes school context in which it was administered, where academic domains are more routinely assessed than creativity. This study examined differences in careless responding across PISA domains and within CT subdomains and facets. Within CT, it also tested whether cognitive and behavioral engagement and self-reported disengagement due to PISA’s low-stakes nature predicted item-level performance. It further investigated whether school-level testing practices moderated the link between disengagement and performance. A secondary analysis of the PISA 2022 data was conducted on 142,564 students who completed at least one CT item. Results showed that careless responding rates were lower for CT items than for core academic domains. Within CT, careless responding also varied across subdomains and facets, with the lowest rates observed for the Social Problem Solving (SoPS) domain and the Generate Creative Ideas (GCI) facet. Performance on CT items was positively associated with both cognitive and behavioral engagement, and negatively associated with disengagement. Finally, among students in schools that routinely use tests for promotion or retention, the negative impact of disengagement on CT performance was weaker. These findings suggest that students took CT tasks at least as seriously as traditional academic ones. We discuss how, despite their potentially more engaging nature, CT assessments are not immune to the same engagement-related issues that affect core domains in low-stakes contexts.