2022 PISA testing likely to be skewed

New Zealand’s students may receive a higher 2022 PISA score than expected due to a skewed sample, says the Ministry of Education.

The Programme for International Student Assessment (PISA) is a two-hour test for 15-year-olds testing literacy, numeracy, science literacy and – in 2022 – creative thinking. It is delivered across the OECD every three years and can be used for international comparisons as well as tracking educational attainment over time.  

Read the Term 4 edition of School News HERE

The results of the 2022 cohort are due out in December, but the Ministry of Education has already released an analysis which states that New Zealand’s scores are likely to be about 10 points higher due to sampling bias resulting from challenges and barriers associated with Covid-19.  

The report states that schools and students are generally randomly selected through a two-stage stratified sampling design. Schools are randomly selected to ensure a representative sample, and from those schools, students are again randomly selected to provide a representative sample of that school’s student body.  

In 2022, 227 English-medium schools with 15-year-old students were randomly selected from Aotearoa New Zealand’s pool. Larger schools were more likely to be selected due to their larger proportion of 15-year-old students. From these schools, 25 – 75 students were selected to participate. However, only 72 percent of selected schools and students participated. Usually, PISA has strict international participation standards of 85 percent of schools and 80 percent of students.  

At the time of selection, RNZ reported that some Principals refused to participate due to Covid-19 related pressure on their students.  

A briefing paper from 2022 to Jan Tinetti said “the reason nearly all declining schools have given relates to Covid-19 – either due to staff shortage/ burdens, student learning loss and/ or the two years of disruption and stress on students and staff.” 

Then Secondary Principals Association president Vaughan Couillault said to RNZ: “Because of what my school has been through in the last 18 months with Covid response and disruption to learning, my senior leadership team and I made the decision that it would be a disruption that we could do without. Because it is quite burdensome to engage in the particular testing that we’re talking about.”