We’re developing New Zealand’s first contextualised value-added model to help evaluate New Zealand secondary schools on a variety of outcome measures. Joel Hernandez explains how using integrated data from ministries, the model aims to adjust for socio-economic factors and determine what effect individual schools have on students’ achievement.
This blog follows on from a recent column on our research and New Zealand’s education system.
Defining School Performance
The role of a school is different for every family.
Some parents want a school where their child will flourish academically. For them grades are a meaningful sign of ‘success’.
Others want a school that helps their child progress into university. Still others want their child to transition into a full-time job or move into industry training straight after college.
Schools can play a long and never-ending list of roles in a student’s life. One of the ways we can improve education success is by measuring school and student performance.
Measures of performance
Some of the roles that a school can play are easy measured and assessed.
Academic results are the easiest to measure because every school in New Zealand records the NCEA results of its students.
Other outcomes measures are more difficult to assess but possible given the level of detail contained in Statistics New Zealand’s Integrated Data Infrastructure (IDI).
Using data in IDI, the New Zealand Initiative is creating a contextualised value-added (VA) model to help evaluate New Zealand secondary schools on a variety of outcome measures – not just average NCEA pass rates. The model does not judge the most important role for a school. That is for parents and students to decide.
In addition to academic outcomes our VA model will also measure post-school outcomes, including progression to tertiary, NEET status (not in employment, education, or training), and benefit uptake after a student has left school.
Many parents in New Zealand want their child to be happy at school in addition to learning English, maths and science. Unfortunately, we don’t have complete data on student happiness for years 11, 12 and 13. To gain information on student happiness, word of mouth within the community and school open days is the only option.
Complement not replacement
Our VA model will not replace current subjective measures of school evaluation. It will not replace Education Review Office reviews. It will not replace open school days. And it will never replace advice from family friends you trust. The model is only an additional evaluation tool for the parents, teachers and children of New Zealand.
The Initiative’s VA model will evaluate schools in a way current evaluation methods do not, that is measure them objectively. The purpose of the VA model is to show variations in school performance after adjusting for the socioeconomic background of each student. It holds constant all the factors that teachers, the PPTA (Post Primary Teachers Association), and anyone involved in education research have ever attributed to differences in student outcomes.
As PPTA President Jack Boyle said, “Everybody knows that the biggest impact on differential outcomes for young people is about socio-economic status and then, beyond that, neuro-diversity, cultural imperatives, etcetera.”
Critics of the VA model argue that it is a one-size-fits-all model. In fact, one of the benefits of the VA model is it can easily be adjusted to different cohorts of students and different school outcomes.
Critics, particularly within the PPTA, also worry that it “could narrow the curriculum and institute performance-based pay for teachers”. The VA model will only evaluate at the school and decile levels because there is no available student-teacher link in the IDI.
Other critics are concerned about student profiling. We argue that our VA model can provide evidence against student profiling. Adjusting the model could provide evidence of success for students who are already negatively profiled based on existing stereotypes and education research in New Zealand.
Finally, compared to VA measures used overseas our contextualised VA model makes significant improvements in that we can control for nearly every aspect of student socioeconomic background and evaluate school performance on a wide variety of outcome measures, not just improvements in standardised literacy and numeracy tests.
This research is genuinely world-leading. Few other countries have access to the kind of data necessary for this kind of research. The Initiative looks forward to sharing our results next year.
Joel has also written about his research for our weekly Insights newsletter – A better way to measure school effectiveness (13 July 2018).