measures that matter achieve education trust adp leadership team meeting september 11, 2008 making...

Measures that MatterMeasures that Matter

AchieveEducation Trust

ADP Leadership Team MeetingSeptember 11, 2008

Making College and Career Readiness the Mission of High Schools


What we set out to do• Develop a framework for states looking to align their

assessment and accountability systems with the goal of college- and career-readiness for all

What we have created• Policymaker’s guide identifying key questions states need to

address along with options and promising practices• Additional tools and papers going deeper in key areas

How we did it• Working with an advisory group of state and national

policymakers, practitioners, and researchers and learning from you


Align High School Standards with the Demands of College and Careers


K-12 standards must be anchored in real world expectations

College ready = career ready Standards must be globally competitive and reflect 21st

century skills Vertical alignment through the grades Standards must reflect tough choices about what’s most

important—fewer, clearer, higher


Assure that Students Enroll in a Course of Study Aligned with College and

Career Readiness Standards


Rigorous course taking matters, especially for low-income and minority students

Graduation course requirements are critical lever for ensuring all students take a rigorous curriculum

Mandatory vs. “opt-out” strategies Monitor course taking patterns carefully Ensure sufficient number of teachers with knowledge and

skills to teach all students more rigorous courses Encourage all students to earn college credits while in

high school


What courses should students be required to take? Will the requirements be mandatory for all students or will

there be an opt-out provision? How will the state ensure consistent rigor across the state? Can the state create multiple but equally rigorous

pathways? How can the state invest in sufficient teacher and student

supports? Don’t let floor become the ceiling—encourage student to

aim higher (e.g., earning college credit in high school)


Measuring Student Learning: A College- and Career-Ready

Assessment System


The Problems: Too many tests and the purposes are disconnected State high school assessments not challenging enough to measure

college and career readiness…so no currency with higher education and the business community

Assessments do not measure the full range of college- and career-ready knowledge and skills (such as research, analysis, critical thinking, and collaboration)

Tests are not generating diagnostic information for educators and students


The Vision: Testing should be streamlined Proficient should mean prepared – anchored in college and

career readiness Tests should measure the full range of college and career

readiness skills –better summative and more performance based assessments

High school tests should open the door for students to higher education and good jobs

Testing should support good teaching


What tests do high school students in your state take right now?

• State? District? Other?

What are the purposes of each?• Instructional improvement• Student credentialing• School accountability

Where could tests be subtracted as new ones are added?


Does your state have a college and career readiness “anchor assessment” in high school?

• If so, is there a statewide readiness score used for placement into credit-bearing courses? Careers?

• If not, what is the strategy for building such a measure?


Benefits Designed to measure state

standards Known quantity in K-12

system Won’t dramatically

increase testing time

Challenges Not designed to measure college and

career readiness Limited credibility with higher

education/employers Most feasible if tests given in 11th

grade

End-of-Grade Tests


Benefits Currency with higher

education and students May not increase testing

burden for many students Common and comparable

results across states

Challenges Not designed to align with state

standards, school curriculum Lack of coherence with rest of

assessment system Require augmentation

College Admissions Tests


Benefits More useful to educators—

aligned to course content Can be used to monitor quality

and consistency of course rigor More flexibility—students

take test when they take course

Challenges May increase overall testing time To assess college/career readiness,

need EOCs in higher level courses Timing of when students take test

can be challenge for postsecondary use

End-of-Course Tests


How does the state ensure that the anchor assessment has credibility with higher education and employers?

• How does test development process need to change?• What studies need to be done to demonstrate a connection

between performance on the test and success in postsecondary?

What type of stakes or incentives can or should be attached to the anchor assessment to make it meaningful for students?


Collaboration to develop and use the tests – in higher education

• California Department of Education (CDE) and California State University (CSU) modified 11th grade assessment

• Placement tests waived for students who score college-ready level and continue to take challenging courses

• Students who do not score at readiness level can get targeted support


Collaboration to develop and use the tests – with employers

• Hawaii K-12, higher education and business leaders working together

• Encouraging students to “opt-in” into college and-career ready course of study

• Tying employer incentives to “opt-in” diploma, which requires Algebra II assessment

Secured commitments to access apprenticeship programs (state carpenters’ union)


Are traditional tests able to measure full set of college and career readiness standards?

Yes and no We can make summative tests better But some important skills better using performance-based

measures (team projects, essays, portfolios, demonstrations, presentations and exhibitions)


Benefits Limits of paper and pencil

test to measure full range of college and career readiness skills

Support richer instruction and deeper student engagement

Challenges Cost Workload burden on

teachers Consistency of scoring

Performance Measures


Is testing at the end of the year or grade sufficient? No…not if goal is to improve teaching and learning How can the state ensure local schools and districts have

access to high quality interim assessments throughout the school year?


Goal is not more testing it’s smarter testing


Provide High Quality Curriculum and Teacher Support Materials


The problem: Curriculum development has been left to individual

districts, schools, and teachers, leading to uneven—and inequitable—course content and instruction.

Source: Prospects (ABT Associates, 1993), in “Prospects: Final Report on Student Outcomes”, PES, DOE, 1997.

Students in Poor Schools Receive ‘A’s for Work That Would Earn ‘Cs’ in Affluent Schools

The Odyssey Ninth GradeHigh-level Assignment

Comparison/Contrast Paper Between Homer's Epic Poem, The Odyssey and the Movie "0 Brother Where Art Thou"

By nature, humans compare and contrast all elements of their world. Why? Because in the juxtaposition of two different things, one can

learn more about each individual thing as well as something about the universal nature of the things being compared.

For this 2-3 page paper you will want to ask yourself the following questions: what larger ideas do you see working in The Odyssey and "0

Brother Where Art Thou"? Do both works treat these issues in the same way? What do the similarities and differences between the works

reveal about the underlying nature of the larger idea?

The Odyssey Ninth GradeLow-level Assignment

Divide class into 3 groups:

Group 1 designs a brochure titled "Odyssey Cruises". The students listen to the story and write down all the places Odysseus visited in his adventures, and list the cost to travel from place to place.

Group 2 draws pictures of each adventure.

Group 3 takes the names of the characters in the story and gods and goddesses in the story and designs a crossword puzzle.

Chicago Public Schools: Many teachers report spending huge amounts of instructional time on ACT practice during 11th-grade core courses

23 23 21

40 3958

0

20

40

60

80

100

Science Math English

Per

cen

t o

f T

each

ers

20 or more hours

12 to 20 hours

Source: Consortium for Chicago School Research, From High School to the Future: ACT Preparation – Too Much, Too Late, Why ACT Scores are Low in Chicago and What It Means for

Schools, May 2008.

“Exams are being used as instructional tools, although they are not designed

for learning. These practices steal instructional time that could be used

for deep, challenging course work that actually would prepare students for

the ACT and for college study.”

Source: Consortium for Chicago School Research, From High School to the Future: ACT Preparation – Too Much, Too Late, Why ACT Scores are Low in Chicago and What It Means for

Schools, May 2008.


The Vision: Getting students into courses with the right names is just

the beginning State policymakers need to provide greater leadership on

the issues of curriculum and instructional supports States should invest in course redesign strategies to

deliver the same content in more accessible and applied ways


Who provides detailed guidance to teachers on what they are expected to teach?

• Who devises the curriculum? Do ALL teachers have access to detailed curriculum frameworks?

• What about well-designed units, lessons and assignments, and links to supplementary resources?

Are teachers in your state expected to “compose” and “conduct”?


How do your teachers in your state know what “good enough” student work looks like?

• Has your state created exemplar or “anchor” assignments, along with scoring rubrics and examples of scored student work? Does EVERY teacher have access to model assignments?

• If these are local responsibilities in your state, how does the state ensure adequate guidance and resources for teachers in districts that lack the capacity to develop high-quality materials?


Is your state prioritizing the “redesign” of priority courses for organizing the same content into different packages?

• A LOT of energy and resources on this effort in higher education.• If CTE is part of the plan, how is the state ensuring these programs

include the academic rigor that students need? • Do CTE programs meet the Perkins “Program of Study”

designation. If not, what’s the plan for assuring alignment?


Getting Everybody Pulling in the Same Direction: An Information and

Accountability System Focused on College- and Career-Readiness


The Problems: Stakeholders do not have access to critical information about

college- and career-readiness. Accountability systems are not based on what it means to be

college- and career-ready, nor are they based on what happens throughout high school.

Accountability systems do not establish expectations for performance that reflect where we need our schools and students to be.

Accountability goals are perceived as something to meet to avoid state interference rather than something meaningful to work toward.


The Vision: Stakeholders need real-time information to support decision-

making. Accountability systems need to reflect key indicators of

progressing toward, achieving, and exceeding college- and career-readiness.

States should set stretch goals that are ambitious and achievable.

Accountability systems should provide incentives for students and schools that meet expectations – and real consequences for continually failing to meet expectations.


What information do stakeholders need to support college- and career-readiness? Are data available to meet these information needs?

• How many students are taking and successfully completing a college- and career-ready course of study? Where are there gaps between groups and districts?

• How are students performing on assessments of college- and career-readiness?

• How many students are graduating, and what happens after they leave high school?


Do accountability indicators reflect the goal of college- and career-readiness?

Measures that MatterMeasures that MatterAlong the way toward college and career readiness

Along the way toward college and career readiness

Exceeding college- and career readiness

Course completion and success

-Timely credit accumulation-Credit recovery

-Successful completion of college- and career-ready course of study

-Participation in AP, IB and dual enrollment

Achievement -Performance on aligned assessments early in high school

-Meeting standards on anchor assessment-Postsecondary remediation rates

-College-level performance on AP and/or IB exams

Attainment -Graduation -Earning a college- and career-ready diploma

-Earning credits in dual enrollment courses-Application to and enrollment in postsecondary


Does the accountability system include stretch goals and progress targets that communicate a clear path from where students, schools, and districts are to where we need them to be?

• Are all schools and students, regardless of where they stand relative to goal of college- and career-readiness, expected to make gains?


80.6% 81.5% 82.4% 83.3% 84.2% 85.1% 85.9% 86.8% 87.7% 88.6% 89.5%

52.9%56.6%

60.2%63.9%

67.5%71.2%

74.9%78.5%

82.2%85.8%

89.5%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Gra

du

ati

on

-Ra

te T

arg

et

Graduation-Rate Progress Targets for Two Tennessee Schools

Data Source: Tennessee Department of Education – Office of Innovation, Improvement and Accountability


Does the accountability system set expectations for multiple indicators while maintaining focus on college- and career-ready graduation?

• This can be accomplished through performance indices that weight the most important indicators and most important levels of performance most heavily, but credit other indicators and other performance levels as well.


Weights for different student outcomes in Louisiana’s accountability system


Do the metrics used in the accountability system evaluate both current-year performance and progress?

• Status: What percentage of students is meeting college- and career-ready standards this year?

• Growth-to-standards: What percentage of students is on trajectory to meet standards by a certain date?

• Improvement: Are schools making sufficient year-to-year improvement in the percentage of students meeting standards to meet the goal by a set date?


What are the positive incentives– for schools and students—to work hard and reach the college- and career-ready level and beyond?

• We cannot motivate excellence only by avoiding sanctions.


The key to this process is COHERENCE.


Making College and Career Readiness the Mission of High Schools

measures that matter achieve education trust adp leadership team meeting september 11, 2008 making...

Documents