summary of findings: reliability student-level reliability (α) grades 3-5 grades 6-12 overall...

Summary of Findings: Reliability

 Student-Level Reliability (α)

Grades 3-5 Grades 6-12Overall Reliability (all items) 0.94 0.96Student Learning 0.90 0.94Students-Centered Environment  0.86 0.90Classroom Community 0.80 0.86Classroom Management 0.75 0.80

 Teacher-Level Reliability (α)

Grades 3-5 Grades 6-12Overall Reliability (all items) 0.97 0.98

Student Learning 0.95 0.97

Students-Centered Environment  0.94 0.96

Classroom Community 0.90 0.94

Classroom Management 0.90 0.91

Relationship between SPS Overall Percent Favorable and Overall Evaluation Rating

What about the survey elements? Classroom Community Classroom Management

Student-Centered Environment Student Learning

Summary of Findings - Validity

• Analyzed 14,539 open-ended responses (3-5 and 6-12)

• We find that not only were the majority of students taking the survey seriously, but that many of the responses were specific and actionable in nature– 98.6% (N=14,341) were considered substantive – 66.3% (N=9,646) were coded as actionable – Moreover, although some subjects and grades were slightly

more likely to garner actionable feedback, in general actionable responses came from students in all grades and subjects

What Students are Saying About Teachers

Overview of Student Feedback for Teachers

Guiding principles for administration

• We know that teachers care about their practice and especially about their students. – They also may experience nervousness and anxiety about the

surveys, and district and school leaders must address those fears.

• By itself, a reliable and valid instrument does not ensure that teachers will receive good feedback.

• Messaging matters! – Engage stakeholders early and often– Make the process as transparent as possible

• Give stakeholders real decision-making power

Engaging Stakeholders

• Forming a planning committee• Building educator investment• Building student understanding and comfort• Informing other stakeholders

Key Decisions

• Use a vendor or manage administration internally?

• Online or paper/pencil survey administration?• When will the survey be administered? • How will students and teachers be sampled?• Include an open-ended question on the survey?• How will survey results be used?

Key Decision: How will survey results be used?

• Consider a “hold-harmless” pilot year• As a formative tool

• Reflect on individual practice• Identify over-arching trends and create strategies to address

them.

• As a part of an evaluation • Used as an artifact for determining ratings for professional

practices• Included as one of several multiple measures

• Sharing teacher-level results

Contact Information

Colorado’s Student Perception Survey Website:

www.colegacy.org/studentsurvey

Amy Farley, Director, Research and Impact

720-502-4723

afarley@colegacy.org

Questions?Elaine Allensworth, Consortium on Chicago School Researchelainea@uchicago.edu

Amy Farley, Colorado Legacy Foundationafarley@colegacy.org

Kendra Wilhelm, Denver Public SchoolsKendra_wilhelm@dpsk12.org

@AYPF_Tweets#aypfevents