Perceptions of academic leaders in New Zealand regarding the functions of moderation of internal assessment: A mixed methods study

Abstract

In formal education and training, internal assessment (in which assessor judgements are made within education organisations) is widely used for summative purposes to contribute to the award of qualifications. In many jurisdictions including New Zealand, organisations that conduct these high-stakes internal assessments are required by regulation to engage in moderation within the organisation and with external quality assurance bodies to quality-assure those assessments. However, policies are rarely implemented directly as intended. Instead, they are enacted by organisations, that is, policies are interpreted and translated, with multiple factors influencing this process. One such factor is the person who takes the role of ‘policy narrator’ and leads the policy interpretation and translation within the organisation. In New Zealand there is further potential for enactment variation because education organisations are largely self-governing, and thus have substantial freedom regarding organisational systems and practices. Moderation is commonly held to have both accountability and improvement purposes. However, it is unknown what policy narrators within New Zealand organisations consider the functions of moderation to be.

This study sought to explore what the academic leaders who are responsible for moderation in New Zealand secondary and tertiary organisations (i.e., those likely to be policy narrators) perceive as the functions of internal moderation and national moderation conducted by the New Zealand Qualifications Authority (NZQA moderation). Further, the study sought to ascertain whether there are any observable differences in perceptions according to organisation type. A pragmatic mixed methods sequential research design was implemented. An online survey instrument was developed informed from interview findings, and then administered using a census approach to collect data (n = 221). Both qualitative and quantitative data analyses were conducted.

Academic leaders were found to believe that moderation functions across multiple embedded contexts, from the immediate assessment event, to organisational and societal contexts. Internal and NZQA moderation were seen to work in the narrowly-focused area of assessment quality, and the broader areas of professional learning, organisational quality assurance, maintaining public and stakeholder confidence, and educational quality (internal moderation only). Instead of subscribing to the dominant improvement and accountability discourses, for the most part academic leaders tended think of moderation in more encompassing ways than the literature suggests.

Respondents from Private Training Enterprises (PTEs) tended to see the organisational quality assurance and educational quality functions as being more important or having a stronger emphasis, and to hold a broader view of moderation functions, than those from schools.

These findings could assist those in organisations to recognise and examine the influence of their own perceptions on practice, and identify opportunities to optimise how their organisations use moderation. The findings enable policy makers to ascertain the degree of alignment between policy intent and enactment, and could inform policy development and communication to the sector. Further, the potential for NZQA to increase the broader and improvement-focused aspects of moderation practice, while maintaining—and enhancing—its accountability focus is highlighted.