summary of findings: reliability student-level reliability (α) grades 3-5 grades 6-12 overall...

11
Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items) 0.94 0.96 Student Learning 0.90 0.94 Teacher-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items) 0.97 0.98 Student Learning 0.95 0.97 Students-Centered Environment 0.94 0.96

Upload: bruce-sparks

Post on 13-Jan-2016

226 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Summary of Findings: Reliability

 Student-Level Reliability (α)

Grades 3-5 Grades 6-12Overall Reliability (all items) 0.94 0.96Student Learning 0.90 0.94Students-Centered Environment  0.86 0.90Classroom Community 0.80 0.86Classroom Management 0.75 0.80

 Teacher-Level Reliability (α)

Grades 3-5 Grades 6-12Overall Reliability (all items) 0.97 0.98

Student Learning 0.95 0.97

Students-Centered Environment  0.94 0.96

Classroom Community 0.90 0.94

Classroom Management 0.90 0.91

Page 2: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Relationship between SPS Overall Percent Favorable and Overall Evaluation Rating

Page 3: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

What about the survey elements? Classroom Community Classroom Management

Student-Centered Environment Student Learning

Page 4: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Summary of Findings - Validity

• Analyzed 14,539 open-ended responses (3-5 and 6-12)

• We find that not only were the majority of students taking the survey seriously, but that many of the responses were specific and actionable in nature– 98.6% (N=14,341) were considered substantive – 66.3% (N=9,646) were coded as actionable – Moreover, although some subjects and grades were slightly

more likely to garner actionable feedback, in general actionable responses came from students in all grades and subjects

Page 5: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

What Students are Saying About Teachers

Overview of Student Feedback for Teachers

Page 6: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Guiding principles for administration

• We know that teachers care about their practice and especially about their students. – They also may experience nervousness and anxiety about the

surveys, and district and school leaders must address those fears.

• By itself, a reliable and valid instrument does not ensure that teachers will receive good feedback.

• Messaging matters! – Engage stakeholders early and often– Make the process as transparent as possible

• Give stakeholders real decision-making power

Page 7: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Engaging Stakeholders

• Forming a planning committee• Building educator investment• Building student understanding and comfort• Informing other stakeholders

Page 8: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Key Decisions

• Use a vendor or manage administration internally?

• Online or paper/pencil survey administration?• When will the survey be administered? • How will students and teachers be sampled?• Include an open-ended question on the survey?• How will survey results be used?

Page 9: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Key Decision: How will survey results be used?

• Consider a “hold-harmless” pilot year• As a formative tool

• Reflect on individual practice• Identify over-arching trends and create strategies to address

them.

• As a part of an evaluation • Used as an artifact for determining ratings for professional

practices• Included as one of several multiple measures

• Sharing teacher-level results

Page 10: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Contact Information

Colorado’s Student Perception Survey Website:

www.colegacy.org/studentsurvey

Amy Farley, Director, Research and Impact

720-502-4723

[email protected]

Page 11: Summary of Findings: Reliability Student-Level Reliability (α) Grades 3-5 Grades 6-12 Overall Reliability (all items)0.940.96 Student Learning0.900.94

Questions?Elaine Allensworth, Consortium on Chicago School [email protected]

Amy Farley, Colorado Legacy [email protected]

Kendra Wilhelm, Denver Public [email protected]

@AYPF_Tweets#aypfevents