Download - Comision Simce Anexo2 Forster
-
8/8/2019 Comision Simce Anexo2 Forster
1/87
ANEXOS
LA EXPERIEN CIA INTERNACIONALEN SISTEMASDE MEDICIN :
ESTUDIODE CASOS
MARGARET FO RSTERG ILBERT A. VALVERDE
Comisin para el Desarrollo y Uso del Sistemade Medicin de la Calidad de la Educacin
DICIEMBRE 2003
-
8/8/2019 Comision Simce Anexo2 Forster
2/87
La Experiencia Internaciona l en Sistemas de Med ici n:
Estudio de Casos
Co misin pa ra el Desarrol lo y Uso del Sistema d e M edic i n
de la C a l idad de la Educac in
ISBN 9 5 6 -2 9 2 -0 8 1 -X
M inisterio de Educacin, Repblica d e Chile
A la me d a 1 3 7 1 , Sa n tia g o
www.mineduc .c l
D ic iembre 2003
-
8/8/2019 Comision Simce Anexo2 Forster
3/87
PRESENTACI N
La Comisin para el Desarrollo y Uso del Sistema de Medicin de la Calidad de la Educacin
recibi entre sus encargos recopilar insumos que permitan conocer los modelos de sistemas
de medicin que se estn utilizando en diferentes pases, y cules son los beneficios y proble-
mas que se asocian a dichos modelos. Ello, con el fin de rescatar aquellos aspectos que se
consideren adecuados para ser incorporados de acuerdo a la realidad de nuestro pas.
Nace as la iniciativa de contar con dos consultores internacionales expertos en el rea de
medicin a nivel de sistemas educativos. Ellos seran los encargados de entregar informacin
para que las recomendaciones de la Comisin tuvieran en cuenta los inconvenientes y solu-
ciones que se han encontrado en otros pases del mundo.
Para cumplir con esta tarea, la Comisin cont con el apoyo de Margaret Forster investiga-
dora del Consejo Australiano para la Investigacin Educativa (Australian Council for
Educational Research, ACER) y de Gilbert Valverde investigador de Albany, Universidad
Estatal de Nueva York.
El trabajo realizado por estos expertos fue dado a conocer en el Seminario Internacional sobre
Sistemas de Medicin y Uso de Resultados1, en el que participaron como expositores. Los
principales planteamientos presentados en dicho seminario fueron completados y
profundizados en documentos elaborados especialmente para la Comisin. Estos documen-
tos son Assessment systems: two case studies, de Margaret Forster y La poltica en evaluaciny currculo ante el desafo de la calidad, de Gilbert Valverde.
A continuacin, se presentan dichos documentos, ellos constituyen los anexos que acompa-
an al documentoEvaluacin de Aprendizajes para una Educacin de Calidad, elaborado por
la Comisin.
-
8/8/2019 Comision Simce Anexo2 Forster
4/87
-
8/8/2019 Comision Simce Anexo2 Forster
5/87
NDICE
PRESENTACI N 05
LA POLTICA EN EVALUACIN Y CURRCULO ANTEELDESAFODELACALIDAD 09
G. VALVERDE
Introduccin 11
TENDENCIASMUNDIALESEN LAO PERACIO N ALIZACIN DECRITERIOSDECALIDAD 13
Fijando criterios para la evaluacin de la calidad 15
ESTUDIO DECASOS 19
Pases Bajos 21
Una prueba longitudinal del aprendizaje para monitorear 2 2
el impacto de polticas
Un modelo mixto y voluntario de pruebas de egreso en 2 3
educacin primaria
Un modelo mixto y obligatorio de pruebas de egreso de 2 3
educacin secundaria
Estado actual de la discusin 2 4
Dos casos en los Estados Unidos 26Nueva York 29
Perfil general del sistema de evaluacin actual 2 9
Sistema actual de pruebas de egreso 3 1
Sistema de responsabilizacin 3 2
Tennessee 34
Perfil general del sistema de evaluacin actual 3 4
El sistema de evaluacin del valor agregado 3 5
Algunas lecciones ilustradas por los casos 37
Obras Consultadas 39
ASSESSMENTSYSTEM S: TWO CASESTUDIES 45
M. FO RSTER
Executive summary 47
Background 51
Introduction 52
-
8/8/2019 Comision Simce Anexo2 Forster
6/87
CASE STUDY 1: W ESTERN AUSTRALIA (WALN A & MSE) 55
Contextual features 57
Geographic, demographic, economic, po li ti ca l and religious context 57
Responsibility for schooling in Australia 5 8
Provision of schooling in Australia 5 9
Funding for schooling in Australia 6 0
C ommon and agreed national goals for schooling in Australia 6 5
The Provision of schooling in W estern Australia 6 6
Student intake 6 7
Funding 6 8
Systemwide assessment 69
Different programs for different purposes 6 9
W ALN A 7 0
M SE 7 3
Educational impact: positives 75
System level monitoring 7 5
School level: use of results and acceptance of program 7 7
Educational impact: concerns 88
CASE STUDY 2: ENGLAND (N ATIONAL CURRICULUM ASSESSMEN T) 93
Contextual features 95
Geographic, demographic, economic, po li ti ca l and religious context 95
Responsibility for schooling in England 9 5
The provision of schooling in England 9 6
Funding for schooling in England 9 6
Agreed goals for schooling in England 9 8
Changed processes of school management 9 9
Target setting 1 00
Student intake 10 0
Funding 10 5
Systemwide assessment 107
Educational impact: positives 109
System level monitoring of standards and initiatives 1 09
School level use of results 1 09
W hat evidence is there that these strategies have improved student learning? 1 15
Educational impact: concerns 117
Final reflections 121
References 122
Useful websites 12 3
-
8/8/2019 Comision Simce Anexo2 Forster
7/87
ASSESSMEN TSYSTEMS:TW OCASESTUDIES
MARGARET FO RSTERAUSTRALIAN COUNCILFOR EDUCATIONAL RESEARCH
THISPAPERHA SBEEN PREPAREDBYTHE AUSTRALIAN COUNCILFOR EDUCATIONAL RESEARCH (ACER)FORTHE CHILEAN M IN ISTRYO F EDUCATION TO INFORMDISCUSSION W ITHTHE N ATIONAL
COMMITTEEFORTHEDEVELO PMENTAN DBETTERUSEO FTHE N ATIONAL ASSESSMEN T SYSTEM .
-
8/8/2019 Comision Simce Anexo2 Forster
8/87
-
8/8/2019 Comision Simce Anexo2 Forster
9/87
4 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES
EXECUTIVE SUMMARY
The management of an education system is a complex and expensive operation, and effective
management demands dependable information on educational outputs. Systemwide (state
or national) assessment programs provide this information through the systematic and regu-
lar measurement of student learning. These programs are designed primarily to investigate
and monitor the health of the system and to improve student learning by providing
information to stakeholders at different levels of the system: policy makers, schools, teachers,
parents, students and the general community.
Of increasing interest internationally is the role that systemwide assessment programs can
play as agents of reform and accountabilityto provide both direction and motivation toschools, teachers, parents and students. In some countries, student achievement data collected
through state assessment programs are used as a measure of schools contributions to student
learning. In some, schools are rewarded or punished on the basis of students results. In
others, public comparisons of schools achievements in the form of league tables are made.
Using assessment programs to drive change directly brings some challenges. First, there can
be unintended negative consequences of assessment driven reform. Second, using assessment
programs as both the instrument of change and as the measure of change makes procedures
open to scrutiny from two perspectives and takes them into the arena of public debate both
about their effectiveness and about their reliability and validity as measures.
The two case studies explored hereWestern Australias state Literacy and Numeracy Assessment
(WALNA) and Monitoring Standards in Education (MSE), programs; and Englands National
Curriculum Assessmentprovide two different approaches to the collection and use of student
achievement data, in the context of reform and accountability demands.
Western Australia has two parallel assessment programs. The WALNA, a full cohort (census) program
administered annually to students in years 3, 5 and 7, provides information on students knowledge,
skills and understandings in numeracy, reading, spelling and writing. The purpose of WALNA is to
provide parents and caregivers with information about the performance of their children in relation
to nationally agreed benchmarks, and in relation to that of other Year 3, 5 or 7 students across
Western Australia; teachers with information on the performance of individual students and classgroups, and on the level of their students literacy and numeracy development in relation to the
performance of other Year 3, 5 or 7 students; and schools with summary information of their overall
performance.
The MSE program is administered annually to a representative sample of students in each of
Years 3, 7 and 10 in one or two of the eight curriculum learning areas. (Aspects of learning areas
-
8/8/2019 Comision Simce Anexo2 Forster
10/87
4 8 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
that are assessed through the WALNA testing program are not assessed through the MSE
program.) The MSE assessments are part of the Departments public accountability procedures
and the results provide a basis for initiating developments that will further improve the outcomesof education in government schools. The purpose of the MSE program is to provide information
to stakeholders about the progress students make in their learning; assist the Department and
schools to monitor and report student performance; and assist the Department and schools to
plan for improvement in student performance.
England has a single systemwide assessment program. The National Curriculum Assessment, a
full cohort (census) program, is administered annually to students at the end of keys stages 1, 2
and 3; that is, at ages seven, 11 and 14. The tests provide information on students knowledge,
skills and understandings in mathematics, reading, writing and spelling and except at key stage 1
in science. The purpose of the tests is to give teachers information about their students attainment,
for use in planning teaching and learning; to provide information to the Government, to localeducation authorities (LEAs) and to the public in general about standards in education nationally
and locally; and to encourage school accountability.
There are many similarities between the Western Australian and the English assessment systems,
in particular the intention to address a number of purposes for assessment and to provide
information of different kinds to a range of stakeholders. Both systems also express a key
intention to impact directly on the work of the classroom teacher.
Both systems:
assess aspects of English (reading, writing, and spelling), and aspects of mathematics at
key stages of schooling; address explicit learning outcomes in the development of their tests;
monitor performance against expectations detailed as described levels of achievement; and
require teachers and schools to use systemwide achievement data in their planning.
The reform and accountability demands addressed directly by the assessment systems are
different, however: The stakes are significantly higher in England than in Western Australia.
The level of expectation is higher in England. When the standards were established in England,
63 per cent of students were above the standard, compared with 85 per cent in Australia.
Target setting has been introduced in England as a management and motivational device.Schools are required to set targets for individual students stating what they are to learn next.
They must also set targets for the school as a whole. One of these targets has to be in terms of
the proportion of students who will achieve level 4 in the National Curriculum tests in
English.
-
8/8/2019 Comision Simce Anexo2 Forster
11/87
4 9A SSESSMENT SYSTEM S: TW O C A S E STUDIES
The test results are high stakes for politicians and their opponents as well as schools, teachers
and students. In 1997 the Government set an overall target that 80 per cent of 11-year old
children should achieve level 4 or higher in English by the year 2002 (an increase of 17 percent over five years). The politician responsible, the Secretary of State for Education and
Employment promised to resign if the target was not met.
The underlying model for the national assessment system is one of competition between
schools with the publication of unadjusted results (league tables). The model assumes that
pressure from parents will assist the government to raise standards.
In other words the central difference between the Western Australian and English systems is
the degree to which the programs are driven by direct accountability pressures. In Western
Australia, although schools need to use the systemwide assessment data in their planning, the
stakes are not high and there are no public school-by-school comparisons. Rather, thegovernment has moved quietly forward to influence teacher practice by sending the message
that all areas of the curriculum are valued, that assessment materials will be made available as
a model for teachers, and that support will be provided to assist schools to use their data more
effectively. The use of two parallel and complementary assessment programs has also assisted
this process. And although this is partly historical (MSE was in existence before the National
agreement to undertake full cohort testing), it is still a strategy worth contemplating.
The difference in reform and accountability pressures also has impacted on the debates
surrounding the assessment systems. In England debate has focused on the validity of
the tests for formative assessment purposes, the distortion of the curriculum and the
misleading use of league tables leading to calls for the publication of value-added measures.This is understandable given the pressure of national targets, and the public nature of
comparisons. Western Australia has so far avoided a heated debate about curriculum
fidelity and is instead exploring issues of equity, and ways to better support teachers in
their classroom practice.
Looking at the changes in school practice in the last couple of years in both countries one
might ask whether the debates are ones now held by primarily by academics and researchers,
rather than by teachers and the community more generally. There is some evidence that in
both countries teachers and schools have accepted the assessment programs and are now
getting on with the job of using data more effectively.
Looking forward and continuing to compare the two systems one might reasonably ask in a
few years time whether the accountability pressures have made enough of a difference to be
worth the initial upheaval. Or whether a less demanding set of expectations can achieve the
same goal of improving teacher practice and student learning outcomes.
-
8/8/2019 Comision Simce Anexo2 Forster
12/87
5 0 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
And taking the big view, as noted by Whetton et al (2000), the experience in England illustrates
that the setting and maintenance of standards is a social and societal process. Assessment is a
social activity that we can understand only by taking account of the social, cultural, economicand political contexts in which it operates. While the setting of standards, for example, relies
on empirical information, the information must be interpreted, and the integrity and expertness
of those making the judgements must be accepted publicly and politically (Gipps, 1999).
-
8/8/2019 Comision Simce Anexo2 Forster
13/87
5 1A SSESSMENT SYSTEM S: TW O C A S E STUDIES
BACKGROUND
The Chilean Ministry of Education has appointed a National Committee to revise the National
Assessment System so that it is better aligned with the purpose of improving education. This
National Committee is providing advice to the Minister of Education in particular about
ways to improve the communication of the assessment systems results to different stakeholders.
To achieve its objectives, the Committee is gathering information about assessment models
used worldwide and considering the benefits and problems of each model with a view to
improving the Chilean assessment system.
This paper contributes to the Committees deliberations by providing two detailed case studies:
Englands National Curriculum Assessment, and the Western Australias (WA) state Monitoringin Education (MSE), and Literacy and Numeracy Assessment (WALNA) programs. The
paper provides a complete description of these assessment systems, including the social and
educational contexts in which they operate, and evaluates the contribution that each makes
to the education system. In addition detailed bibliographic references and possible sources of
information about the systems and tests are provided.
The paper draws on information from websites, research papers, public documents, discussions,
and previous research undertaken in preparation for the publication of ACERs Policy makers
guide to systemwide assessment programs (Forster, 2001) and Principals guide to the use of
systemwide assessment data (in final draft form).
-
8/8/2019 Comision Simce Anexo2 Forster
14/87
5 2 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
IN TRODUCTION
The management of an education system is a complex and expensive operation, and effective
management demands dependable information on educational outputs. Systemwide (state
or national) assessment programs provide this information through the systematic and regu-
lar measurement of student learning. These programs are designed to investigate and moni-
tor the health of the system and to improve student learning by providing information to
stakeholders at different levels of the system:
policy makers with information to monitor standards over time, to monitor the impact
of particular programs, and to make decisions about resource allocation; and
schools (principals, councils) and teachers with information about whole school, class
and individual pupil performance that they can use to make decisions about resourceallocation and to support learning in the classroom.
Full-cohort (census) programs also provide:
parents with information about their childs progress to assist them to make decisions
about the best ways to support their child; and
students with information about their progress to assist them to take an active role in
their own learning.
Of increasing interest internationally is the role that systemwide assessment programs can
play as agents of reform and accountabilityto provide both direction and motivation to
schools, teachers, parents and students. In some countries, student achievement data collectedthrough state assessment programs are used as a measure of schools contributions to student
learning. In some, schools are rewarded or punished on the basis of students results. In
others, public comparisons of schools achievements in the form of league tables are made.
The theoretical appeal of this intention is easy to understand. The influence of assessment
programs in motivating educational change is well known. When results matter, (when the
stakes are high) assessment programs influence what is taught by reinforcing aspects of the
curriculum; and they engage schools, teachers, and the community in educational debate.
The political appeal is understandable also:
Compared with reforms such as targeting instructional time, professional development forteachers, and reducing class sizes, state assessment programs are relatively inexpensive. The
assessments also can be mandated (unlike changes in classroom practice), can be rapidly
implemented, and have a public visibility (Linn, 1995).
However, using assessment programs to change the education system more generally brings
some challenges. First, there can be unintended negative consequences of assessment driven
-
8/8/2019 Comision Simce Anexo2 Forster
15/87
5 3A SSESSMENT SYSTEM S: TW O C A S E STUDIES
reform. For example, the impact of the minimum competency testing movement in the United
States is well known (Marion & Sheinker,1999). Second, using assessment programs as both
the instrument of change and as the measure of change makes procedures open to scrutiny fromtwo perspectives and takes them into the arena of public debate both about their effectiveness
and also their reliability and validity as measures (Whetton et al, 2000, p.5).
This paper explores two systemwide assessment systems: Western Australias (WA) state
Monitoring in Education (MSE), and Literacy and Numeracy Assessment (WALNA)
programs; and Englands National Curriculum Assessment. Despite their different political,
social and educational contexts, there are many similarities in the Western Australian and the
English assessment systems, in particular the intention to address a number of purposes for
assessment and to provide information of different kinds to a range of stakeholders. Both
systems also express a key intention to impact directly on the work of the classroom teacher.
The reform and accountability demands addressed directly by the assessment systems aredifferent, however with the stakes significantly higher in England than in Western Australia.
-
8/8/2019 Comision Simce Anexo2 Forster
16/87
-
8/8/2019 Comision Simce Anexo2 Forster
17/87
CASE STUDY 1
W ESTERN AUSTRALIA(W ALNA & MSE)
-
8/8/2019 Comision Simce Anexo2 Forster
18/87
-
8/8/2019 Comision Simce Anexo2 Forster
19/87
5 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES
CONTEXTUALFEATURES1
Geographic, demographic, economic, politicaland religious context of Australian schooling
Australia is a large island continent located close to south east Asia and the South Pacific. The
land mass totals 7.7 million square kilometres and much of the interior is extremely arid. The
vast majority of people live in the major cities on the coasts.
The total population in 1996 was approximately 18,311,000, with an annual population
growth rate of 1.2 per cent. Around 80 per cent of the population live on narrow coastal
strips covering less than four per cent of the total land area. As a result, Australia has a high
level of urbanisation.Aboriginal people and Torres Strait Islander people, the indigenous
population, account for around 1.6 per cent of the total population and are spread throughout
urban and rural areas. There is considerable diversity of cultures within both Aboriginal and
Torres Strait Islander communities and particular efforts are made to provide for their special
educational needs and to assist in cultural maintenance.
In recent years, Australia has had one of the highest levels of immigration of any developed
country. By 1996, for example, approximately 24 per cent of the population had been born
overseas. Since the early 1970s, as the population has grown, the proportion of the population
of school age has decreased.The International Labour Organisation unemployment rate for
Australia (persons aged 15 years and over) for 1999 was 7.2 per cent (7.3 per cent for men;
7.1 per cent for women). Australia is secular, having a wide variety of religious affiliations,
including Christians (mainly Catholics and Anglicans), Buddhists, Muslims, Hindus and
Jews. The Catholic church is the largest provider of school education, other than the State.
The Australian political system includes an elected federal government and elected governments
of the six States (New South Wales, Victoria, Queensland, Western Australia, South Austra-
lia, Tasmania) and two self-governing Territories (Northern Territory and the Australian Ca-
pital Territory). Voting is compulsory.
1 This section of the paper draws directly from two texts: the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and Youth Affairs
(MCEETYA) website http://www.curriculum.edu.au and the Archive of the International Review of Curriculum and Assessment Frameworks Project (INCA) http://www.inca.org.uk/
-
8/8/2019 Comision Simce Anexo2 Forster
20/87
5 8 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
Responsibil ity for schooling in Australia
The Constitution of Australia allocates responsibility for school education to State and Territory
governments, all of whom provide and manage government schools as well as supporting
non-government schools. Government schools operate under the direct responsibility of the
relevant State or Territory Minister, while non-government schools are established and operate
under conditions determined by State or Territory government registration authorities. Many
non-government schools have some religious affiliation, most commonly with the Catholic
Church; 19.9 per cent of all students and 63.6 per cent of non-government students were
enrolled in Catholic schools in 2001.
The Ministers of Education in the States and Territories have responsibility for primary and
secondary schooling, student enrolment policies, curriculum content, course accreditation
and certification procedures, and student assessment. Each of the State/Territory Governments
is responsible for determining teacher qualifications, establishing and paying teachers salaries,
recruiting and appointing teachers in government schools and supplying buildings, equipment
and materials.
In cooperation with the States and Territories, the Federal government through the
Commonwealth Department of Education, Science and Training (DEST) plays a significant
role in addressing equity and quality issues through general recurrent, capital and specific
purpose programs. In addition, it has specific responsibilities for Aboriginal people and
migrants and is responsible for international relations in education.
The key national body is the Ministerial Council for Education, Employment, Training and
Youth Affairs (MCEETYA) which comprises the Commonwealth, State, Territory and New Zealand
Ministers with responsibility for the portfolios of education, employment, training and youth
affairs, with Papua New Guinea having observer status. MCEETYA meets on a biannual basis to
discuss issues of mutual interest and to coordinate collaborative policies and approaches and generally
to facilitate the exchange of information on education in Australia and overseas.MCEETYA is
supported by taskforces that are convened as needed for particular tasks. These taskforces have
prescribed timeframes and reporting arrangements and are reviewed annually.
MCEETYA has established a range of consultative mechanisms at Federal and State levels to
obtain the views of the non-government school sector and, whenever possible, extends membership
of its working parties and committees to the major national bodies representing the non-government
employing authorities: the National Catholic Education Commission and the National Council
of Independent Schools Associations. Regular consultation also takes place with the principal
national bodies representing parents teachers and the business sector. Two national research and
development companies also facilitate cooperative initiatives in schooling: the Australian Council
for Educational Research (ACER), an independent research body, and the Curriculum Corporation
which is wholly funded by Commonwealth and State Ministers of Education.
-
8/8/2019 Comision Simce Anexo2 Forster
21/87
5 9A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Provision of schooling in Australia
Schooling in Australia is compulsory for children aged 6 to 15 (16 in the state of Tasmania)
years of age. However, the majority of children start school before they are six and remain at
school beyond the age of 15. In most States, children start full-time schooling at around the
age of five, when they enrol in a kindergarten or preparatory year. Commonly, the majority
of these students will have already had some part-time school or preschool experience. After
the preparatory year, primary education lasts for either six or seven years, and secondary
schooling for five or six years, depending on the State. Students usually commence secondary
school at about age 12.
In 2001, there were approximately 1.9 million primary school students and 1.36 million
secondary school students in Australian schools. Approximately 72.4 per cent of primary
students were enrolled in government schools, and 63.7 per cent of secondary students in
government schools. Most government schools are coeducational, but a significant number
of non-government schools are single-sex schools.
Some features of the structure of Australian schooling in 2001 were as follows:
There were 9596 schools in Australia.
There were 3.268 million full-time students in Australian schools.
The proportion of students enrolled in non-government schools continued to rise in
all States and Territories. In 2001, 31.2 per cent of students were enrolled in non-
government schools compared to 30.8 per cent in 2000.
There were 115,465 Indigenous students enrolled in Australian schools in 2001. This
represented 3.53 per cent of the total school population and was a significant increase
from 2.13 per cent in 1990.
In 2001, 49.9 per cent of Australias 15-19-year-olds were full-time students at school.
The largest year cohort of primary school students was in Year 1 (138,576 students)
and the largest year cohort of secondary students was in Year 8 (131,507 students).
There were 221,927 teaching staff (in full-time equivalents) employed in Australian
schools, which resulted in an average of 14.8 students per teacher in government schools,
15.9 per teacher in Catholic schools and 12.8 per teacher in independent schools.
Females comprised 78.7 per cent of the teaching staff in primary schools and 54.8 per cent
in secondary schools, compared with 78.3 per cent and 54.4 per cent respectively in 2000.
-
8/8/2019 Comision Simce Anexo2 Forster
22/87
6 0 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
Funding for schooling in Australia
Within each State and Territory, ministers, departments, statutory authorities and individual
schools (particularly in the case of non-government schools) variously determine policies and
practices and provide resources for infrastructure in areas such as curriculum, course
accreditation, student assessment and certification, resource allocation and utilisation, and
teacher employment and professional development.
Through DEST, the Commonwealth provides funding to both government and non-government
school authorities to support agreed priorities and strategies. The overall result is that government
schools receive the majority of their government funding from State and Territory governments
and less from the Commonwealth, while non-government schools receive the majority of their
government funding from the Commonwealth, and less from the relevant State or Territory.
Funding is provided also from fees, charges and levies paid by students and their parents or
caregivers and private donations and income, including that derived from fundraising.
Figure 1 Public funding for school educationSource: 2000 National Report on Schooling in Australia Amounts in AUD
-
8/8/2019 Comision Simce Anexo2 Forster
23/87
-
8/8/2019 Comision Simce Anexo2 Forster
24/87
6 2 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
In the financial year 1999-2000 the total per capita expenditure on government schools was
$AUD 6 358 ($5,687 for primary students; $7,416 for secondary students.)
Non-government schools derive their income from fees and donations, and State and
Commonwealth government grants. Per capita expenditure on non-government schools in
the financial year 1999-2000 is shown below in Table 1.
Source: Derived from data provided by Commonwealth DETYA , 2000 National Report on Schooling in Australia
Of the $4.78 billion of grants for schools spent under the States Grants (Primary and Secondary
Education Assistance) Act 1996 and the Indigenous Education (Supplementary Assistance) Act
1989, the government sector received $1.77 billion and the non-government sector $3 billion.
A further $10 million was spent on joint national programs benefiting both sectors.
Table 2 below shows how funding from the States Grants (Primary and Secondary Education
Assistance) Act and the Indigenous Education Supplementary Assistance Act was distributed
between government and non-government authorities, by State.
-
8/8/2019 Comision Simce Anexo2 Forster
25/87
-
8/8/2019 Comision Simce Anexo2 Forster
26/87
6 4 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
Note: Expenditure with respect to a certain program year may continue in relation to that year in future years. Source: 2000National Report on Schooling in Australia, Commonwealth DETYA (2000 program year cash expenditure as at 30 June 2000)
-
8/8/2019 Comision Simce Anexo2 Forster
27/87
6 5A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Common and agreed national goals for schooling in Australia
In 1989 the Education ministers from the States and Territories and the national
(Commonwealth) Government published a set of Common and Agreed (National) Goals
for Schooling in Australia. Known as the Hobart Declaration, these goals provided a structure
for cooperation between schools, States, Territories and the Commonwealth, named eight
Key Learning Areas (the arts; English; health and physical education (HPE); languages other
than English (LOTE); mathematics; science; studies of society and environment (SOSE);
and technology) and described ten national goals for schooling.
In 1999 the State, Territory and Commonwealth Ministers for Education endorsed a revised
set of National Goals, the Adelaide Declaration2 See and affirmed their commitment to
national reporting on comparable educational outcomes in literacy; numeracy; student
participation, retention and completion; vocational education and training in schools; science;
and information technology. (The Ministers also noted the need to develop performance
indicators for civics and citizenship education, and enterprise education.)
The literacy and numeracy goal is described as follow: students should have attained the
skills of numeracy and English literacy; such that, every student should be numerate, able to
read, write, spell and communicate at an appropriate level. To achieve this goal Ministers
agreed to support:
assessment of all students by their teachers as early as possible in the first years of
schooling;
early intervention strategies for those students identified as having difficulty;
the development of agreed benchmarks for years 3, 5, 7 and 9, against which all
student achievement in these years could be measured;
the measurement of students progress against these benchmarks using rigorous State-
based assessment procedures;
progress towards national reporting on student achievement against the benchmarks,
with reporting commencing in 1999 within the framework of the annual National
Report on Schooling in Australia; and
professional development for teachers to support the key elements of the plan.
By 2000 performance standards for literacy and numeracy at years 3, 5 and 7 were developed
and the Ministers agreed that benchmarking for years 9 or 10 be postponed pending findings
from the OECD Program for International Student Assessment (PISA) project. The
benchmarks represent the minimum acceptable standard of numeracy without which a student
will have difficulty making sufficient progress at school. They were developed with reference
to levels of achievement as demonstrated in national surveys and State assessment programs
through an extensive consultation process with stakeholders and with experts in the areas of
numeracy and educational measurement and trial testing in classrooms in all States and
Territories. Existing State-based programs are used for assessment against the national
2 See http://www.detya.gov.au/schools/adelaide/index.htm
-
8/8/2019 Comision Simce Anexo2 Forster
28/87
6 6 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
benchmarks and a nationally agreed procedure to equate State and Territory tests ensures that
reporting of student achievement data is comparable by States and Territories.
Professional development for teachers is an integral part of the National Literacy and Numeracy
Plan, as it is recognised that the classroom teacher is the major determinant of the literacy
learning of students. During 2000, there was considerable professional development related to
the assessment and intervention programs described above. In particular, teachers were assisted
to interpret data from assessment programs and devise programs based on their findings.
Many teachers were involved in professional development programs associated with the
introduction of particular intervention programs. As many schools had instituted literacy
teams, a number of the professional development initiatives were directed towards the team
leaders. A train-the-trainer approach was common in such programs.
Another professional development strategy was the establishment of literacy networks among
teachers. The increasing use of modern communication technology contributed significantly
to the extent and effectiveness of such strategies. Many authorities also provided opportunities
for teachers to engage in postgraduate, accredited study in literacy teaching and learning.
The provision of schooling in Western Australia
In Western Australia under the provisions of the School Education Act 1999, every person
living permanently in Western Australia and aged between six and 15 years must be enrolledat a government school or a non-government school, or receive approved home tuition.
Government schools provide schooling during the period of compulsory attendance to any
person who wishes to enrol. In addition, access to optional schooling is available for four-
year-old and five-year-old children and persons aged over 15.
The Education Department is the largest employer in Western Australia employing the
equivalent of 25 000 fulltime staff and accounting for a quarter of the State budget. The
Department is responsible for the education of 258 000 students at 771 schools across the
state from metropolitan Perth (the capital) to rural and remote centres. Over 1300 students
receive approved home education.
In the Perth metropolitan area and major rural and remote centres, government primary
schools provide sessional kindergarten, optional full-time pre-primary and full-time primary
education (Kindergarten to Year 7 or Primary to Year 7), and high, middle and senior high
schools and senior campuses full-time secondary (7-12, 8-10, 8-12 or 11-12) education. A
small number of early childhood education centres and junior primary schools offer Kinder-
garten to Year 2, or Kindergarten to Year 3 education respectively.
-
8/8/2019 Comision Simce Anexo2 Forster
29/87
6 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Outside the major rural and remote centres, smaller primary schools, district high schools
and remote community schools are provided. Kindergarten to Year 10 (and, in some cases,
Kindergarten to Year 12) schooling is delivered by district high schools supported bytelematics learning systems which integrate the use of personal computers, interactive
television, the Internet, fax and telephone. The remote community schools deliver Kinder-
garten to Year 10 education with the support of communications technology to Aboriginal
children, for many of whom English is a second language or second dialect. A number of
campuses deliver specialised services to male youths experiencing social or behavioural
difficulties. Education support schools and centres cater for students with disabilities for
whom specialised school environments are most appropriate. Students with mild to severe
intellectual disabilities attending rural or remote schools are supported by the provision of
equipment and learning materials and, for their teachers, professional development and
assistance with the design of individualised education programs. Delivery of schooling to
geographically-isolated students involves the Schools of Isolated and Distance Education(SIDE); small Kindergarten to Year 7 and Kindergarten to Year 10 schools in remote areas;
and a combination of face-to-face, interactive and traditional distance education teaching
and learning.
Student intake
Government schools in Western Australia cannot exclude students from enrolment within
their intake area. The School Education Act 1999 1/Jan/2001 states under section 78
Enrolment of children of compulsory school age at local-intake school.
A child of compulsory school age is entitled to be enrolled at a local-intake school if his
or her usual place of residence is in the intake area for that school; and an appropriate
educational programme is available for the child at that school.
A child of compulsory school age whose usual place of residence is not in the intake
area for a local-intake school is entitled to be enrolled at that school if there is available
for the child at that school an appropriate educational programme and classroom
accommodation, and the enrolment would conform with any other criteria prescribed
by the regulations for the purposes of the subsection of the Act.
Schools must give priority to students who are within their intake area. If there are spaces left,
they can take students from outside the school intake boundaries. Schools can also apply to
have special programs recognised and funded by the State (for example, a program for
gifted and talented students). Schools delivering these programs can set aside a number of
places that can be filled from within or outside of the school intake boundaries.
-
8/8/2019 Comision Simce Anexo2 Forster
30/87
-
8/8/2019 Comision Simce Anexo2 Forster
31/87
6 9A SSESSMENT SYSTEM S: TW O C A S E STUDIES
SYSTEMW IDE ASSESSMENT3
Different programs different purposes
The Western Australian Department of Education conducts two systemwide assessment
programs:
the Western Australian Literacy and Numeracy Assessment (WALNA) program; and
the Monitoring Standards in Education (MSE) program.
The WALNA is a full cohort (census) program administered annually to students in years 3,
5 and 7. The assessments address explicit curriculum learning outcomes in numeracy, reading,
spelling and writing.
The purpose of WALNA is to provide:
parents and caregivers with information about the performance of their children in
relation to nationally agreed benchmarks in numeracy, reading, spelling and writing,
and in relation to that of other Year 3, 5 or 7 students across Western Australia;
teachers with information, not only on the performance of individual students, but also on
that of class groups, and on the level of their students literacy and numeracy development
in relation to the performance of other Year 3, 5 or 7 students. This information can be used
in conjunction with their own class records in future teaching and learning programs;
schools with a report showing summary information of their overall school performance.
This data is useful additional information in the schools identification of students needs.
Through the, schools are also provided with assistance in interpreting their results for
school improvement and accountability purposes.
The MSE program is administered annually to a representative sample of students in
each of Years 3, 7 and 10 are tested in one or two of the eight learning areas. Aspects of
learning areas that are assessed through the WALNA testing program are not assessed
through the Random Sample assessment program. The assessments are part of the
Departments public accountability procedures and the results provide a basis for initiating
developments that will further improve the outcomes of education in government schools.
The purpose of the MSE program is to: provide information to stakeholders about the progress students make in their learning;
assist the Department and schools to monitor and report student performance4; and
assist the Department and schools to plan for improvement in student performance.
3 This section of the paper draws information primarily from three sources the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm information provided on the Western Australian Department of Education websites for the two system-level assessment programshttp://www.eddept.wa.edu.au/
walna/ and http://www.eddept.wa.edu.au/mse/ A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western
Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003).4 The Department also produces School Assessment Materials for the key learning areas. These materials assist schools to fulfill their accountability requirements
as well as provide diagnostic information to teachers on the strengths and weaknesses in individual student learning. This information can help teachers plan forstudent progress through the curriculum. Some of these documents are available in PDF on the Department website: http://www.eddept.wa.edu.au/mse/materials.html
-
8/8/2019 Comision Simce Anexo2 Forster
32/87
7 0 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
WALNA
The WALNA full cohort (census) assessment program commenced in 1998 with the assessment
of Year 3 students in literacy (defined as reading, writing and spelling) and expanded in 1999
to include Year 5 students and the assessment of numeracy. In 2001 the program was further
extended to include Year 7 students. In 2002 almost all students in Government schools in
Year 3, Year 5 and Year 7 (approximately 61,500 students) undertook the WALNA tests in
literacy and numeracy5. In addition, around 22 000 students from Catholic and independent
schools now also participate in the testing program. The annual budget for WALNA is $AUD
1.3 million. Around 40% of the budget is spent in the hand-marking operation of the writing,
the open ended reading and numeracy questions and the hand marking of spelling.
The WALNA program was developed in response to the National Literacy and Numeracy
Plan. The tests, which address both outcomes in the WA Outcome and Standards Framework
(1998)6 and the National Benchmarks, are administered annually in August (the school year
runs from February to December).
The timeline for the 2002 WALNA program for all WA schools involved government and
private is provided in Table 3.
(Source: Murdoch University and Estill & Associates, 2003).
5 Parents/caregivers can exercise their right to withdraw their child or children from the testing program. This can be carried out through the completion of anexemption form, available from schools prior to testing. Exemptions are also granted, at the school level, on the grounds of disability or impairment. Decisions toexempt students are made by the principal, after discussion with the classroom teacher and school-based specialists, and with the signed agreement of the parents/caregivers. Some English as Second Language (ESL) students in mainstream classes, as well as those in Intensive Language Centres, may also be granted exemptions.In 2002, less than 1% of exemptions were lodged on the grounds of temporary or permanent disability, and 0.2% of children were withdrawn from the testingprogram by their parents.
6 All Department of Education schools are required to plan and structure their learning and teaching programs using this Curriculum Framework.
-
8/8/2019 Comision Simce Anexo2 Forster
33/87
7 1A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Extensive consultation with classroom teachers is a feature of the development of the assessment
materials and the tests are written to cater for the diverse range of students in Western Australian
schools to ensure that there is no systematic bias associated with such factors as gender,culture or geographic location. The numeracy and reading components of the test take between
30 and 60 minutes, and the writing component takes 65 minutes (though children only
spend up to 35 minutes writing). The spelling component takes about 20 minutes. In most
cases, the regular teacher conducts the assessment tasks as part of the normal school program.
Samples of the WALNA literacy and numeracy tests can be found on the WALNA website
at:http://www.eddept.wa.edu.au/walna/pdfs/sample%20magazine.pdf and
http://www.eddept.wa.edu.au/walna/pdfs/sample%20test.pdf
A team of experienced teachers completes all marking and to ensure reliability, markers undergo
extensive training. To further enhance reliability, all marking is conducted at a central location.Markers work in groups under the direction of a team leader, and reliability scripts are administered
at each session. Each group leader monitors the marking, and provides advice as necessary.
The tests are designed to:
provide an indication of how students are performing in relation to the National
Benchmarks,
measure student performance in relation to the Outcomes and Standards Framework and
provide comparative information on student performance in relation to State perfor-
mance.
Data are analysed using the Rasch Measurement Model. This model allows the abilities ofstudents and the difficulty of the assessment tasks they have completed in an area of learning
to be brought onto a single scale. Each scale provides two pieces of information: a continuum
of skills and understandings in the area of learning, based on the assessment items, arranged
in order of increasing difficulty, and the distribution of students in relation to the tasks.
This method of scaling allows inferences to be made about the probability of the success of
any student on all individual items. Each item can also be mapped to the levels of achievement
described in the WA standards framework (the Student Outcome Statements) making it
possible to identify areas of the sale that relate to each level. In turn, this makes it possible to
draw inferences about the level of achievement of groups of students. (Appendix 2 provides
an explanation of this process.)
-
8/8/2019 Comision Simce Anexo2 Forster
34/87
-
8/8/2019 Comision Simce Anexo2 Forster
35/87
7 3A SSESSMENT SYSTEM S: TW O C A S E STUDIES
MSE
The MSE program began in 1990. Each year a random sample of approximately ten percent of
students in each of years 3, 7 and 10 in government schools participate in the program8. Schools
are notified of their involvement in the weeks prior to the assessments. Schools decide the days
and times to administer the assessments during the specified week (week 8 of term 3).
Assessments occur in all key learning areas, as well as in the social outcomes of schooling, and
address the significant learning outcomes (knowledge, understandings, skills and processes)
that students are expected to develop as they move through the compulsory years of schooling.
These learning outcomes are described in the Outcome and Standards Framework. In most
cases, all sequenced strands of a learning area are assessed. For example, in the 1997 Random
Sample assessment of Science, Year 3, 7 and 10 students were assessed in Life and Living,
Energy and Change, Natural and Processed Materials, Earth and Beyond and Investigating
Scientifically. Some learning areas, such as Mathematics and English, are assessed more regularly
than others so that trends in performance to be closely monitored. For example, aspects of
the English learning area have been tested in 1992, 1995, 1997, 1999 and 2001.
The annual budget for the Random Sample program is almost $AUD 1 million. This budget
covers the development or administration of three learning areas (two being assessed and at
least one under development), and the production and on-going provision of school release
to schools.
The MSE program uses rigorous procedures to train markers to make consistent judgements
of student performances. Markers are trained to use standardised marking guides, in
conjunction with student work samples, to judge student work.
As with the WALNA program, MSE data are analysed using the Rasch Measurement Model
and single vertical scales from Year 3 through to Year 10 provide the basis for reporting.
Student performance is reported using the eight levels of achievement described in the Student
Outcome Statements. Aggregated information on the performance of students in Years 3, 7
and 10 is reported. Within these year groups, information on boys, girls, Aboriginal and
Torres Strait Islander (ATSI) students and Language Backgrounds Other Than English
(LBOTE) students is also reported. For each of these groups, the mean performance and the
range of performance across the state are described in terms of the levels of the Outcomes
Framework.
MSE produces public reports for the Western Australian government education system . The
reports present information descriptively, in table form and through graphs. In most reports,
samples of students work from the assessments are included, to exemplify level achievement.
Examples of MSE reports are included in Appendix 2.
8 Year 3 marks the end of the early childhood phase of schooling, Year 7 the end of the middle childhood phase and Year 10 the end of the compulsory years ofschooling, and the end of the early adolescent phase.
-
8/8/2019 Comision Simce Anexo2 Forster
36/87
7 4 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
MSE releases a selection of the random samples materials free of charge to WA Government
schools. The intention is that the materials will provide models of good assessment tasks, assist
teachers to set their own assessment criteria, and develop teachers capacity to make judgementsabout students work using the Student Outcome Statements. Depending on the nature of the
learning area, the released materials consist of stimulus materials (print/audio/video), student
test booklets, teacher administration instructions, marking keys with moderation instructions,
guidelines for analysing results, and performance profiles or progress maps.
At the classroom level, teachers use the results to:
diagnose individual students strengths and weaknesses in terms of actual skills and
understandings that the student does or does not demonstrate on the assessment;
determine the strengths and weaknesses of the class as a whole;
determine the strengths and weaknesses in the teaching of aspects of the curriculum;
obtain an indication of level achievement in the substrands described in the CurriculumFramework
At the whole school level, school administrators use the results to:
infer level achievement in the substrands described in the Curriculum Framework;
make comparisons with the state mean;
make comparisons with the state range;
track year cohorts longitudinally;
track groups of students over time;
report to the school community;
inform school development plans; and
provide continuity for students between schools (primary/primary and primary/high)
School administrators and teachers are reminded that the MSE assessments do not cover the
entire curriculum and that the results provide only one source of information. They are also
reminded that the administration instructions need to be followed carefully in order to maximise
standardisation and that the materials cannot be used to compare teachers and schools.
-
8/8/2019 Comision Simce Anexo2 Forster
37/87
7 5A SSESSMENT SYSTEM S: TW O C A S E STUDIES
EDUCATIONALIMPACT: POSITIVES9
System level monitoring
Since 1990, MSE data have facilitated the monitoring of student achievement at a system level.
Now, the combined data from WALNA and MSE allow for the monitoring against an expected
standard. Data from WALNA have also provided a check on the random sample findings.
The WA Department reports two major findings in relation to the standards:
1. Some minor fluctuations are evident in the data from year to year and in the different
strands but the overall trend emerging from the data is one of slight improvement.
2. The percentages of students achieving the standards in reading, writing and spelling
vary from 1998 to 2002. Variations of this sort appear in all large scale testing programs,
and the apparent improvements or declines should be interpreted cautiously. The
observed trend of slight improvement over the years of testing is encouraging.
The Department also reports the following findings in relation to previous MSE findings:
1. Girls continue to do better than boys in literacy at all years. The difference in perfor-
mance between boys and girls increases from Year 3 to Year 5 and from Year 5 to Year 7
and is most evident in writing.
2. Girls and boys numeracy results are comparable.
3. Aboriginal students literacy performance has shown a trend of improvement since
1998. While Aboriginal students performance is still below that of the total population,
there are indications that their rate of improvement in reading and writing is slightly
greater than that of the general population.
9 This section of the paper draws information and direct text primarily from the following sources the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm information provided on the Western Australian Department of Education website http://www.eddept.wa.edu.au/ A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western
Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003). information provided electronically by Helen Wildy, Associate Professor of Educational Leadership, School of Education, Murdoch University an article written by Helen Wildy reviewing the work of the Data Club (Wildy, 2003) the Western Australian School Accountability Framework, 2002 http://www.eddept.wa.edu.au/regframe/index.cfm discussions with Jocelyn Cook, Manager Educational Measurement, Standards and Accountability, Education Department Western Australia
-
8/8/2019 Comision Simce Anexo2 Forster
38/87
7 6 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
The following three tables show the percentage of students reaching the benchmark standards
at Year 3, Year 5 and Year 7.
-
8/8/2019 Comision Simce Anexo2 Forster
39/87
7 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES
School level: use of results and acceptance of the program
At the school level, evidence of the impact of WALNA data is confirmed by the work of the Data
Club, by an independent review, and by the requirements of the school accountability framework
The Data Club
The Data Club assists schools to use their WALNA information through an annual half-day
workshop program that the principal and one other person with curriculum responsibilityfrom each participating school attend. The Club began in 2000 as a pilot project with two
hundred volunteer schools and grew over the following three years until in 2003, 508 schools
(of a total of 612 government schools) were participating10.
In advance of the workshops WALNA data are re-analysed to provide participants with box
and whisker representations of school level performance on each of the four WALNA strands:
Reading, Writing, Spelling and Numeracy. Comparisons are made over time, as the data are
accumulated, and regression analyses are used to show expected growth or value added in
school performance in a given strand over time. Standardised residuals are used to indicate
the relative amount of change.
Workshop participants are introduced first to the box and whisker method of representing
data as well as to regression analysis, standardized residuals, confidence limits and examples
of valid and invalid inference. In the second part of the workshop participants are given
envelopes containing their processed data and are supported as they work through tasks
designed to explore their data.
10 The project began as a partnership between Edith Cowan University and the Western Australian Department of Education and Training, supported by theCommonwealth Department of Education, Science and Training to develop schools capacity to make performance judgements in relation to the NationalLiteracy and Numeracy Plan. From 2003 the project was retained, managed and funded entirely within the WA Department.
-
8/8/2019 Comision Simce Anexo2 Forster
40/87
-
8/8/2019 Comision Simce Anexo2 Forster
41/87
7 9A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Independent review of WALNA
In 1999, in response to criticism of the WALNA test, an evaluation survey was conducted by
the Education Department. The survey report indicated that teachers criticisms of the length
and difficulty of the test were not supported by parents who regarded the test information as
an appropriate mechanism to help raise literacy standards. Four years later there was anecdotal
evidence that the WALNA program had become more accepted by teachers as the longitudinal
benefits to parents, teachers and schools become apparent. In light of this anecdotal evidence,
the Department of Education commissioned an independent evaluation study of the 2002
WALNA (Murdoch University & Estill and Associates, 2003).
The study addressed two main questions: What are the attitudes of Year 3, 5 & 7 teachers
and parents to WALNA five years after its commencement, and how have they changed since1999? How is the information from WALNA used by schools?
To answer the first question, the evaluation re-administered a revised form of the teacher and
parent questionnaires from the 1999 WALA evaluation to allow comparison with the data
from that year. The second question was addressed by developing a new questionnaire for
principals, and by supplementing the teacher questionnaire with additional items.
In all, 458 principals, 1458 teachers and 1414 parents responded to the survey. Principals responses
were analysed as a total group and according to school type and size. Teachers responses were
analysed as a total group and according to school type and size, year level taught, and the number
of years teaching experience. Parents responses were analysed as a total group and according toschool type and year level of their child.
Overall, principals reported that they made considerable use of the WALNA information, found
it useful for teachers and parents, and had adopted it into their schools performance measurement
and planning activities. However, principals in Remote Community Schools and Education
Support Schools did not have the same positive perceptions of the WALNA information.
Generally, teachers had positive perceptions about the WALNA program. The majority
understood the purposes of the program, felt they received sufficient information and support
to administer the tests and analyse the results, and were not overly concerned that the tests
were too stressful or too long for most students. The exception to this is teachers in RemoteCommunity Schools and Education Support Centres, who had consistently less positive
perceptions. Compared to teachers perceptions in the 1999 survey, the current survey shows
a general improvement in teacher perceptions.
-
8/8/2019 Comision Simce Anexo2 Forster
42/87
8 0 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
The perceptions of parents and caregivers of children who undertook the 2003 WALNA tests
were generally positive and had improved since the 1999 survey in nearly all areas.
Nearly all the parents agreed that the purpose of the test was to provide them with information
about the achievements of their children (90%) and to assist schools to provide better literacy
and numeracy teaching for all students (95%). The large majority also agreed that it provided
information about student weaknesses (81%). The majority (60%) thought that the test was
used to compare different educational systems; however, they disagreed (57%) that the purpose
of the test was to judge the quality of the teachers. They were evenly divided over whether a
purpose of the test was to judge the quality of schools. Nearly all parents (96%) agreed that the
report gave them information about the achievement of their child in relation to the benchmark
and that the report was clear and understandable (89%). About half the respondents (52%)
agreed that the school had provided additional information which helped them understand the
report. The large majority agreed that the report confirmed what their childs regular schoolreports were saying (74%) but thought that the report gave them additional information (73%).
The large majority of parents/caregivers did not believe that their children were unusually stressed
in the lead-up to the test (87%) or when completing the test (90%). Neither did they think that
the test was too difficult (80%) or too long (83%) for them. Rather they believed that their
child enjoyed participating in the test (74%) and were pleased with their test results (82%).
Only a few (12%) thought that their child was upset with their result. While they were divided
as to whether the test results had encouraged their child to work harder (54% agreed with this
statement), almost all (95%) disagreed that it had had a discouraging effect on them.
Both in 1999 and 2003 parents/caregivers and teachers generally agreed that the purpose ofthe test was to provide parents with information about their children, and highlight student
weaknesses.
-
8/8/2019 Comision Simce Anexo2 Forster
43/87
8 1A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Role of the test in assisting schools toprovide better literacy teaching for all students
The 1999 report pointed out that nearly all parents/caregivers (92%) believed that the purpose
of the test is to help provide better literacy teaching and that the information from the test
program will help to raise literacy standards across the State. In contrast, a small majority of
teachers (58%) felt the results would be useful in this way. This difference was reduced
dramatically in 2003: as shown below, 73% of teachers valued the test information more as a
source of diagnostic information about their students, and used it to identify students at risk
and who required extension.
QUESTION: ASSISTSCHOOLSTOPROVIDEBETTERLITERACYTEACHINGFORALLSTUDENTS
-
8/8/2019 Comision Simce Anexo2 Forster
44/87
8 2 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
The quality of the information provided
In 1999, the majority of parents believed that the reports confirmed regular school reports
but nonetheless supplied them with additional information. On the contrary, teachers surveyed
in 1999 agreed that the results were in line with school-based tests but only a minority of
teachers (37%) agreed that they provided valuable diagnostic information about their students.
This pattern has changed over the intervening period to the point where the majority of
teachers (62%) now value the test information, using it to identify students at risk or requiring
extension. A greater percentage of parents also believe the tests provide useful additional
information (62% to 73%).
QUESTION: THEREPORTGAVEMEADDITIONALINFORMATIONNO TAVAILABLEINTHEREGULARSCHOOLREPORT
QUESTION: THETESTRESULTSPROVIDEDMEWITHVALUABLEDIAGNOSTICINFORMATIONABOUTMYSTUDENTS
-
8/8/2019 Comision Simce Anexo2 Forster
45/87
8 3A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Reaction of students to the test
In 1999, considerably more teachers than parents/caregivers were of the opinion that students
were stressed before and during the test. In 2002, the proportion of teachers with this opinion
was reduced, though still considerably greater than that of parents/caregivers.
QUESTION: INTHELEAD-UPTOTHETESTMYCHILDWASUNUSUALLYSTRESSEDABOUTPARTICIPATIN GINTHETEST
QUESTION: INTHELEAD-UPTOTHETESTSOMEOFMYSTUDENTSWEREUNUSUALLYSTRESSEDABOUTPARTICIPATIN GINTHETEST
-
8/8/2019 Comision Simce Anexo2 Forster
46/87
8 4 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
Confidentiality of results
In 1999, a large majority of teachers (70%) were concerned about the confidentiality of results,
a view not shared by many parents (23%). This concern was shown to be considerably reduced
in 2003, dramatically for teachers (decline to 38%) but also for parents (decline to 14%).
Overall, in the areas where there were differences between teachers and parents in 1999, the
differences have been dramatically reduced and both groups have more positive perceptions.
Compared to 1998, teachers in particular, have become more accepting of WALNA. Parents
and caregivers were generally positive and had more positive views in all areas than werereported in the 1999 survey. With familiarity over the years and the help of the Data Club,
the WALNA data is accepted as a tool which can be used diversely, for example, by individual
teachers for judging students or by the schools for assisting with curriculum planning.
Feedback from principals, teachers and parents from Remote Community Schools and
Education Support Centres indicates they view the WALNA tests are not as appropriate for
QUESTION: I AMCONCERNEDTHATMYCHILDSRESULTSWILLNO TBEKEPTCONFIDENTIAL
QUESTION: I HAVECONCERNSABOUTTHECONFIDENTIALITYOFTESTRESULTS
-
8/8/2019 Comision Simce Anexo2 Forster
47/87
8 5A SSESSMENT SYSTEM S: TW O C A S E STUDIES
students in these schools as they are for students in mainstream schools. This is because they
see the tests as more stressful for the students and more disruptive to their learning.
Nevertheless, there is some support for the view that despite this, WALNA can still play apositive role in measuring progress in improving student outcomes over the years.
The 2002 survey results demonstrate that the WALNA is now accepted by the three key stakeholder
groups as part of the normal cycle of reporting on student numeracy and literacy outcomes.
The School Accountability Framework
The School Accountability Framework sets out the requirements for all schools in relation to
accountability and review.
All schools are required to produce, in partnership with their school community, a school
plan setting out their objectives, priorities, major initiatives and evaluation measures. They
are required also to assess their performance in terms of standards of student achievement
and the effectiveness of the school; and make available to the public and to the District
Director a School Report that describes the schools performance. Central to this activity is
the use of achievement data collected through the systemwide assessment programs. School
staff are accountable to the principal and school principals are accountable to the District
Director for the performance of the school.
The Framework is built on five fundamental commitments: high standards; effectiveness;
quality teaching; quality information for parents about the standards being achieved by their
schools; information for the wider community about the standards being achieved across the
government schools system. The accountability policy is based on five key concepts.
1. Accountability involves responding to information about performance.
That is, accepting responsibility for the achievement of outcomes and taking appropriate
action to improve performance. Key accountability questions for schools include:
What are we trying to achieve?
How well are we achieving it?
How can we improve?
-
8/8/2019 Comision Simce Anexo2 Forster
48/87
8 6 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
2. Accountability is linked to outcomes.
The core purpose of schools is to enable the success of each and every student. Schools
therefore must be able to demonstrate to themselves and to the communities they servethat what they do makes a difference to student learning. Since schools do not have
control over all the variables that impact upon student achievement, they cannot be
held accountable for it in an absolute sense. However, because the school can make a
difference to the level of achievement of the students, schools are accountable for
maximising the difference they make in terms of student outcomes.
3. Demonstrating accountability is a professional responsibility of school staff.
Schools need to take responsibility for demonstrating their effectiveness to others. That
is, accountability is located within the professional responsibility of the school staff, by
creating an expectation that staff demonstrate the quality of their work as part of the
expression of their professionalism.
4. Rigorous self-assessment is at the heart of effective accountability.
While schools and systems vary in the approach they take to school accountability, the
success of any approach ultimately depends on how well the school reviews its own per-
formance. Effective internal review involves the school gathering information about the
levels of student achievement, analysing and judging the adequacy of those levels,
identifying strengths and weaknesses in its overall performance, and assessing which aspects
of the schools operation should be changed to generate improved performance.
5. Schools are accountable to their communities and to the Director-General.
The Framework acknowledges the two-way accountability of schools. Schools listenand respond to their local communitys needs and provide information about the schools
performance. They also respond to systemic priorities and requirements and provide
an assurance that every government school is meeting accountability requirements.
The planning model used by a school should reflect the needs of the school. That is, so
long as the plan enables the school to respond to the communitys needs, accommodate
Departmental initiatives, and demonstrate its accountability, schools have flexibility in
terms of the planning model they adopt.
The Framework makes the role of school self-assessment clear and the changed context of
teacher assessment explicit. In the past teachers have not used a common set of outcomes on
which to base their judgements of student performance. Establishing the comparability ofteachers judgements against the Outcomes and Standards Framework requires a great deal of
moderation activity within and across schools. Teachers will need to have opportunities to
discuss and share examples of what constitutes a particular level of achievement by a student.
Many schools already use Monitoring Standards in Education (MSE) test materials as part of
a process of moderating teachers judgements about levels of student achievement. The process
of fine-tuning teachers judgements will also be assisted by annual literacy and numeracy
-
8/8/2019 Comision Simce Anexo2 Forster
49/87
8 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES
benchmark testing for year 3, year 5 and year 7 students. As well as monitoring the perfor-
mance of students across the school and at particular year levels, school staff also need to
know about the performance of particular groups of students: perhaps the girls are doingbetter than the boys or the performance of the Aboriginal students is different from that of
the other students.
Analysing achievement data will assist schools to target their improvement efforts at the
points where improvement is most needed. Having gathered some information about student
achievement levels, a key question in any self-assessment is whether that level of performance
is up to expectations. Reaching an answer to this question involves comparing the schools
performance with a standard of some kind. The standard might be a State average; it might
be based on other schools with similar student populations: it might be a national benchmark;
or it might be the schools own performance in previous years. Schools will be assisted in this
process and will be provided with information about the performance of like schools onnational literacy and numeracy benchmark testing.
Having reviewed their performance in terms of student achievement, school staff also need to
assess whether the school is operating as effectively as it could so that the maximum value is
added to the students learning. They are also required to produce a School Report that gives
parents and members of local communities a clear sense of the standards being achieved and
the schools effectiveness in reaching the objectives set out in their school plans. Parents need
to have information reported to them that shows the progress being made in the agreed
priority areas of the school plan. Any information that could identify individual students
must not be published in School Reports. In general, information about school performance
should be communicated in such a way as to ensure that the maximum amount of informationis placed in the public domain while avoiding simplistic league tables based on narrow measures
of school performance. The kinds of information that are to be included in the report are
detailed in Appendix 4.
As a final accountability measure, reviews of schools by District Directors provide assurance
to all stakeholders that schools have conducted rigorous and comprehensive self-assessments,
reported the required outcomes in their School Reports and responded with appropriate
improvement plans. This validation enables those with an interest in a school to have confidence
in the quality of the schools review and reporting processes. It also provides schools with
valuable feedback about their performance and directions for improvement.
-
8/8/2019 Comision Simce Anexo2 Forster
50/87
8 8 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
EDUCATIONALIMPACT: CONCERNS12
A number of issues relating to the identification, measurement and presentation of data have
emerged in Australia. Below is a summary of the issues that are currently under consideration.
Ethical principles underpinning reporting
The performance information that is collected and published has the potential to affect greatly
the public perception of school education as well as to influence the decision-making for
government and non-government schools. It is essential, therefore, that the presentation of this
information is underpinned by a clear set of ethical principles. During 2000, this matter was
considered by one of national working groups, which proposed that the fundamental ethical
principle should be truth in reporting. Such a principle requires that the information reported
should be accurate and comprehensive, reporting should not be selective or partial, and all
relevant information should be reported. This principle should also be followed when data are
presented in graphical and tabular forms. Two other supporting ethical principles were proposed:
the communitys right to information; and
the avoidance of unwarranted harm to members of the community.
The first of these is based on the assumption that the public has the right to access information
that has been collected by government for public purposes. However, this principle has to be
considered in the light of the second, which acknowledges that, because of the potential for
harm to occur to individuals and institutions, there can be no absolute right of publication.
Ensuring privacy
Most jurisdictions have in place legislation covering privacy and/or freedom of information. Typically,
this legislation refers to matters such as the collection of personal information, solicitation of
information, its storage and security as well as access to the information. The Australian NationalReport operates within these provisions and also acknowledges that some jurisdictions have agreements
with particular organisations and sections of the community that need to be respected. The over-
riding protocol being observed is that no information should be reported that allows the identification
of individual students or schools. Where information is provided about individual schools, teachers
or students (for example, in some case studies) appropriate approvals are sought prior to publication.
12 This section of the paper draws directly from Appendix 4 the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and
Youth Affairs (MCEETYA) website http://www.curriculum.edu.au
-
8/8/2019 Comision Simce Anexo2 Forster
51/87
8 9A SSESSMENT SYSTEM S: TW O C A S E STUDIES
Selection and reporting of performance measures
The Ministerial Council on Education, Employment, Training and Youth Affairs has defined
key performance measures as: A set of measures, limited in number and strategic in orientation,
that provides nationally comparable data on aspects of performance critical to the monitoring
of progress against the National Goals for Schooling in the Twenty-first Century.
Initial efforts to develop such data have focused on the more measurable aspects of students
performance. However, it is expected that performance measures will eventually be developed
to cover the broad range of educational outcomes, even though this might be conceptually and
operationally more difficult in some areas. Some general issues surrounding the reporting of
performance measures include the following:
Time-series data are preferable to single-year data and are included where possible with
descriptive comments regarding trends.
Caution should be taken whenever data involves comparisons between jurisdictions,
sectors or institutions, as extraneous factors can influence the data and lead to readers
drawing false conclusions.
Comparative data of this kind should draw the attention of the reader to all relevant factors.
Data will also be provided, where possible, for students according to gender,
socioeconomic status and geographic location.
International studies may provide good information about the performance of Australian
students and when used, they need to be accompanied by appropriate contextual information.
Data quality
It is intended that only data that are deemed to be of high quality will be published. This
requires clear data specifications which, while not always being published, must be available
from the organisations responsible for data collection. It is important that estimates of statistical
uncertainty are published with the performance measures and not in a separate appendix.
The usefulness of data will be enhanced by the publication of contextual information that
enables the reader to take a considered and critical view of the measure. Similarly, when
information is necessary to explain the data, caveats should be included and be placed as near
as possible to the data.
-
8/8/2019 Comision Simce Anexo2 Forster
52/87
9 0 A SSESSMENT SYSTEMS: TW O C A S E STUDIES
Census (full cohort) versus sample data
The latest edition of the National Report on Schooling In Australia contains data derived from
whole-of-cohort census measurement as well as data obtained from samples. Both kinds of
measurement can provide useful information for nationally comparable reporting. Whole-cohort
testing is likely to be both more expensive and intrusive than sample testing and these factors may
make sample testing the preferred option. It is also recognised that sample testing may make
possible the use of richer assessment tasks than are likely to be possible in census testing. However,
sample size must be sufficient to allow reporting by jurisdictions and by sub-groups. Stratified
samples are usually constructed to ensure adequate numbers of students in some sub-groups.
A similar situation exists when using survey data. Census data can be disaggregated to report
by jurisdictions and by sub-groups but a