presentation for directors of special education (12/17/13) michael flicek, ed.d. education...
TRANSCRIPT
PRESENTATION FOR DIRECTORS OF SPECIAL EDUCATION
(12 /17 /13)
MICHAEL FLICEK, ED.D.EDUCATION ACCOUNTABILITY CONSULTANT
2013-14 Wyoming School Accountability
WAEA School Performance Levels
Exceeding ExpectationsMeeting ExpectationsPartially Meeting ExpectationsNot Meeting Expectations
2
Indicators used to Identify School Performance Level
Schools with grades 3 through 8 Achievement Growth Equity
Schools with grades 9 through 12 Achievement Readiness Equity
3
Achievement – Grades 3-8
Assessments used in 2014 PAWS reading – Grades 3-8 PAWS math – Grades 3-8 PAWS science – Grades 4 & 8 SAWS – Grades 3, 5 &7
4
Achievement – Grade 11
Assessment used in 2014 ACT Subject-Area Tests
Reading Mathematics Science Combined English/Writing
5
2014 Standard Setting
PAWS & ACT Subject Area TestsNew student performance levels to be
establishedAlignment with Wyoming State Standards
(i.e. CCSS)Expected to be more rigorous than current
performance levels
Some Business Rules
Minimum n for all indicators = 10 When fewer than 10 students look back is applied Look back 1 year first, then a second year when
needed Small school review when fewer than 2 indicators
meet minimum nFull Academic Year Only
October 1st to mid point of testing window
Grade 3 – 8 Model
Illustration of Computation of a Grade 3-8School Achievement Score
9
Content Count of Tested Scores
Count of Proficient
Scores
School Achievement
Score
Math 80 65
Reading 80 60
Writing 40 25
Science 20 12
Column Totals 220 162 162/220 = 73.6%
Professional Judgment Panel (PJP)
A representative group of 27 to 30 people Representing groups prescribed by statute
Selected by the State Board of Education
10
PJP Major Tasks
Determined the cut points for school scores on each indicator that determine if schools are:Exceeding TargetsMeeting TargetsBelow Targets
11
Student Growth – Grades 4-8
Growth in reading and in mathEach student will have a student
growth percentile (SGP)Same grade in schoolSimilar test scores in previous yearsScores from 1 to 99Excellent indicator for special
education program improvement
12
School Growth – Grades 4-8
For school accountabilityFor accountability
School median SGP (MGP) The SGP that ½ of students at the school
scored above and ½ of students at the school scored below
For school & program improvementSpecial Education median SGP (MGP)
District overall Within a school Suppression rules applied to reports
13
Equity – Consolidated Subgroup
Current subgroups performance will continue to be reported
For Wyoming accountability, however, a consolidated subgroup will be used
The consolidated subgroup = all students who scored below proficient on the previous year’s reading test and/or math test
14
Equity – Grades 4-8
Adequate Growth Percentile (AGP) The SGP on this year’s test, the student needs to be on track
for being proficient within 3 years or by the end of grade 8 These are computed for all students in grades 4 through 8
with more than one previous PAWS testSchool’s Equity Score
The percent of students at the school who are considered on track to become proficient within 3 years (i.e., for whom their SGP equaled or exceeded their AGP)
Another excellent indicator for special education What % of special education students are on-track for
proficiency
15
High School Model
High School Achievement Indicator
Alternative to Percent Proficient
Fluctuations in scale scores from year-to-year (i.e., documentation on WDE website)
Percent proficient cut-scores subject to substantial fluctuation unrelated to school performance
Alternative standardized score approach More stable More sensitive to changes due to school performance
Student Standardized Scores
Identify a baseline year (2013) Compute statewide mean score and standard deviation for that year For example, on the math subject area test:
Mean = 19.6 Standard deviation = 4.5
Compute a standardized score for each student (state mean – student score)/ state standard deviation Assume student score is 21 (21 – 19.6) = 1.4 1.4/4.5 = 0.31 “The student scored 31% of a standard deviation above the baseline
mean score” These scores will be negative when the student score is below the
state baseline mean score
School Achievement Score
The mean student standardized score at the school multiplied by 100 and rounded to a whole number The mean is for all subject area tests combined Each school score will be a whole number Most school will have scores will be between -100 and
+100 “A school with a score of 28 had a mean student score
that was 28% of a standard deviation above the baseline year state mean score”
A mean of the special education subgroup would be helpful
High School Equity Indicator
POLICY OBJECTIVE: TO ENCOURAGE A FOCUS ON IMPROVING PERFORMANCE OF
THE MOST HIGH-RISK STUDENTS
High School Consolidated Subgroup
Criteria for consolidated subgroup membership Current grade 11 students with
Grade 10 PLAN scores on 2013 subject area tests Below 17 on the math test (bottom 37% of scores)
and/or Below 16 on the reading test (bottom 33% of scores)
Schools were notified on November 5th in a WDE Assessment Update about these criteria for the 2014 high school consolidated subgroup
Equity Score Computation
Illustration: student standardized score computation State ACT math baseline mean score = 19.6 State ACT math baseline year standard deviation =
4.5 Assume a student’s ACT math score is 17 (17 – 19.6) = -2.6 -2.6/4.5 = -0.58 This student’s score was 58% of a standard deviation
below the state baseline year mean scoreHigh School Equity Score = the median
student standardized score for the consolidated subgroup for reading and math combined
Readiness – Grades 9-12
• Performance on ACT Suite (Explore, Plan & ACT)
• Graduation index• Grade 9 credits earned• Hathaway eligibility
24
ACT Suite – Average Index Score for all Tested Students
Composite Score RangesWyoming
ACT Readiness
Levels
ACT Explore Grade 9
ACT Plan Grade 10
ACT Test Grade 11
Index Points
Level 1 1-14 1-15 1-16 20
Level 2 15-17 16-18 17-20 50
Level 3 18-20 19-21 21-24 80
Level 4 21-25 22-32 25-36 100• Aligned with Hathaway Scholarship eligibility cut-points• Each student receives an index score• The average of the index scores for all students will be the school score
25
Graduation Index
Criteria Numbers Student Result Points*
1 Diploma Earned in Four Years
100
2 Diploma Earned in More than Four
Years
100
3 Continued Enrollment***
50
4 Noncompleter 0
• The index points were established by the PJP
26
Grade 9 Credits Earned
The percent of students who completed grade 9 with one fourth of the credits required to receive a diploma
27
Hathaway Scholarship Eligibility Index
Student Eligibility Level Points
Honors 100
Performance 90
Opportunity 80
Provisional 70
Not Eligible 0
• The school’s score will be the mean of the student points for the graduating class at the school• The school receives an index score for each graduate• The average of the index scores for all students will be the school score
28
Illustration of Total School Readiness Score
Subindicator
Hypothetical Score for a
School
Example Subindicator
Weight(School Score
* Weight)
ACT Suite Index
55 .20 11
Graduation Index
67 .25 17
Grade 9 Credits
72 .10 7
Hathaway Eligibility
80 .45 36
School Readiness Score (Sum of Subindicator Weighted Scores) =
71
29
Achievement Below
Achievement Meeting
Achievement Exceeding
Growth Below NOT PARTIALLY PARTIALLY
Equity BelowGrowth Meeting PARTIALLY MEETING MEETING
Growth
Exceeding PARTIALLY MEETING MEETING
Growth Below PARTIALLY MEETING MEETING
Equity MeetingGrowth Meeting PARTIALLY MEETING MEETING
Growth
Exceeding PARTIALLY MEETING EXCEEDING
Growth Below PARTIALLY MEETING MEETING Equity
ExceedingGrowth Meeting PARTIALLY MEETING EXCEEDING
Growth
Exceeding MEETING MEETING EXCEEDING
Decision Table for Schools with Three Indicators for Grades 3-8
30
Performance Level Descriptions(For Schools with Grades 3-8)
EXCEEDING EXPECTATIONS: This category is reserved for schools considered models of performance. These schools demonstrated high achievement and exceeded target on at least one other performance indicator – equity or growth – while meeting target on the other indicator.
MEETING EXPECTATIONS: Schools in this category demonstrated performance that met or exceeded target on multiple performance indicators. These schools typically had acceptable or better levels of achievement, student growth, and/or in promoting equity for students with below-proficient achievement.
PARTIALLY MEETING EXPECTATIONS: Schools in this category performed below target on multiple performance indicators or were below target in achievement while failing to exceed target in the other indicator(s). Many schools in this category showed acceptable performance in promoting equity based on growth for students with below-proficient achievement and/or met target for student growth from year to year.
NOT MEETING EXPECTATIONS: Schools in this category had unacceptable performance on all indicators. For schools in this category, improvement is an urgent priority. These schools have low levels of achievement, demonstrate below-target growth, and fall short of producing academic improvement for below-proficient students that will move them toward proficiency.
PJP 2013 Version
31
Decision Table for Schools with Two Indicators for Grades 3-8
32
Achievement Below
Achievement Meeting
Achievement Exceeding
Growth Below NOT PARTIALLY PARTIALLY
Growth Meeting PARTIALLY MEETING MEETING
Growth Exceeding PARTIALLY MEETING EXCEEDING
Achievement Below
Achievement Meeting
Achievement Exceeding
Readiness
Below NOT PARTIALLY PARTIALLY
Equity BelowReadiness Meeting PARTIALLY MEETING MEETING
Readiness Exceeding PARTIALLY MEETING MEETING
Readiness
Below PARTIALLY MEETING MEETING
Equity MeetingReadiness Meeting PARTIALLY MEETING MEETING
Readiness Exceeding PARTIALLY MEETING EXCEEDING
Readiness
Below PARTIALLY MEETING MEETING Equity
ExceedingReadiness Meeting PARTIALLY MEETING EXCEEDING
Readiness Exceeding PARTIALLY MEETING EXCEEDING
Decision Table for Schools with Three Indicators for High Schools
33
Performance Level Descriptions(For High Schools)
EXCEEDING EXPECTATIONS: This category is reserved for schools considered models
of performance. These schools demonstrated high achievement and exceeded target on at least one other performance indicator – equity or readiness – while meeting target on the other indicator.
MEETING EXPECTATIONS: Schools in this category demonstrated performance that met or exceeded target on multiple performance indicators. These schools typically had acceptable or better levels of achievement, student readiness, and/or in promoting equity for students with below-proficient achievement.
PARTIALLY MEETING EXPECTATIONS: Schools in this category demonstrated either unacceptable levels of achievement or were below target on improving the achievement of below-proficient students and on graduation rate and tested readiness. Many schools in this category showed acceptable performance in promoting equity based on growth for low achieving students and/or met target for student readiness.
NOT MEETING EXPECTATIONS: Schools in this category had unacceptable performance on all indicators. For schools in this category, improvement is an urgent priority. These schools have low levels of achievement, fall short of targets on graduation and tested readiness, and have large achievement gaps that show little or no improvement.
PJP 2013 Version
Decision Table for High Schools with Two Indicators
35
Achievement Below
Achievement Meeting
Achievement Exceeding
Readiness Below NOT PARTIALLY PARTIALLY
Readiness Meeting PARTIALLY MEETING MEETING
Readiness Exceeding
PARTIALLY MEETING EXCEEDING
Participation Rate Impact
Schools with grades 3-8• All schools had participation rates of 98% or higher in
2013• One small school was docked a performance level for
having less than 95% participation when a prior year was included in an attempt to meet the minimum n requirement This school went from “meeting” to “partially meeting”
High School Participation Rate
11 of 84 high schools had participation rates on the ACT suite of between 90% and <95% • 3 of these were already “not meeting expectations”• 4 dropped from “meeting” to “partially meeting”• 4 dropped from “partially meeting” to “not meeting”
Participation Rate “Not Met”
12 high schools had less than 90% participation rate on the ACT suite of tests• 8 of these were already “not meeting” • 3 dropped from “partially meeting” to “not meeting” • 1 dropped from “meeting” to “not meeting”
The Result
Grades 3-8 did well on participation rate13 of 84 high schools (16%) had lower
performance levels because of poor participation rate on the ACT suite of tests
An additional 10 of 84 high schools (12%) had less than 95% participation rate on the ACT Suite but were already “not meeting”
28% of high schools had participation Rate Problems.
