communicating about evaluations with student growth measures ccsso topical meeting raleigh, nc sept....
TRANSCRIPT
COMMUNICATING ABOUT EVALUATIONS WITH STUDENT GROWTH MEASURES
CCSSO TOPICALMEETINGRALEIGH, NC
Sept. 19, 2013
www.ksaplus.com
2
The View from 30,000 Feet
www.ksaplus.com
Distribution
3
Content
www.ksaplus.com
You need to translate
我們非常歡迎您的孩子在經過一夏天特別忙碌的準備之後重新回到學校。在這個夏天及過去幾天內，當我在學校視察時，看到老師和學校的主管們一直在孜孜不倦地工作著，確保您的孩子在接下來的一學年中能夠享受到充滿魅力和挑戰的課堂生
活。
“
”
Estamos encantados de dar de nuevo la bienvenida a la escuela a su hijo tras un verano especialmente cargado de preparativos. Durante mis visitas a las escuelas a lo largo del verano y en los últimos días, he comprobado cómo profesores y directores trabajaban incansablemente para garantizarles un año escolar repleto de actividades interesantes y estimulantes para su hijo.
“
”
ايک پر طور خصوصی ہوئے بهرے سے تياريوںکا بچے کے ہمآپ بعد گرماکے موسم! مصروف
پ&رجوش ہوئے کرتے استقبال پر واپسی ميں سکولکچهدنوں پچهلے اور گرماميں موسم! پورے ہيں۔
کو قائدين کے سکول اور اساتذہ نے ميں،ميںليے کے بنانے يہيقينی ہے ديکها کامکرتے انتهکاور مشغوليت سال تعليمی کا بچے کے کہآپ
ہو پ&ر سے اسباق “محرک
”
www.ksaplus.com
5
3 Key Questions
Who?
What?
Why?
www.ksaplus.com
6
SEA colleagues (“family
first”)
Teachers/Principals
Legislators
Media
Who?
www.ksaplus.com
7
New
Different
Better
What?
www.ksaplus.com
8
What’s in it for me?
Why?
www.ksaplus.com
9
Students at the Center
www.ksaplus.com
10
All Roads Lead to Student SuccessEffective Teaching
Standards/curriculum
Tests
School leadership
Data/Accountability
www.ksaplus.com
11
Evaluations
Measures
All Roads Lead to Student Success
11
Effective Teaching
Standards/curriculum
Tests
School leadership
Data/Accountability
Support
Feedback
How to Discuss Student Growth Models
www.ksaplus.com
13
Why are evaluations such a hot-button issue in the first place? Tying student
achievement data to
evaluations embodies
the fears and anxieties
that many teachers have
about reform and their
profession. Exacerbating factors:
• Scary math• High stakes• Little perceived value• Misinformation
Concerns about
testing
Concerns about
mechanizationof teaching
Concerns aboutlack of control and input
Concerns about
data-drivenaccountability
Evals
Source: TNTP
www.ksaplus.com
14
Value-Added Model: Lessons Learned
Hold realistic expectations
The goal is not to make teachers diehard fans of VA, but to ensure they understand the basics and accept VA as one of multiple measures.
Emphasize function within the larger evaluation system
Emphasize that VA is just one part of the puzzle. Focus on its unique role as an objective measure of student learning that can balance more subjective measures, like principal observations.
Acknowledge shortcomings
Be honest about the limitations of value-added and have a plan for addressing them. Glossing over the challenges will only increase skepticism.
Be prepared for misinformation
Concerns often stem from misinformation about what VA is and how it will be used. Know the myths and have succinct responses ready.
Stay out of the weeds
Offer a detailed explanation for those who are interested, but stay focused on the big picture.
Plan for glitches Implementation will expose problems. Create and communicate an easy way for teachers to check for and report issues or errors.
Source: TNTP
www.ksaplus.com
Critical Themes We’re all here to help
students learn. Of all the tools we can
use to measure a teacher’s impact on student learning, value-added is the most sophisticated.
Value-added isn’t perfect, but it doesn’t have to be perfect to be useful.
Teaching is complex and can’t possibly be captured in a single measure.
Students of teachers with high VA ratings don’t just do well on tests.
Source: TNTP
www.ksaplus.com
16
Be Prepared
Explain the What
Explain the Why
Explain the How
Anticipate and Combat Resistance
Be Prepared for Questions
www.ksaplus.com
17
What It IsValue-added analysis uses standardized test scores to determine a teacher’s impact on student growth per year
What It Isn’tUnlike other measures, it takes into account each student’s starting point based on background and previous performance
What It Gives Us
It allows us to better understand what impact the teacher had
1. Explain the What
www.ksaplus.com
18
Use Visuals To Show How It Works
Source: The Los Angeles Times
www.ksaplus.com
19
Show How It Captures Gains, Not Just Scores
While Student A meets State standards, his value-added score reveals decreasing performance.
Conversely, Student B does not meet State standards, but his
value-added score reveals improved performance.
Source: Chicago Public Schools
www.ksaplus.com
20
Show How It Works for Students with Different Starting Points
Source: DC Public Schools
www.ksaplus.com
21
ATTAINMENT MODELWhich gardener has the tallest tree?
Gardener A
Gardener B
72 in.61 in.
www.ksaplus.com
22
GAIN MODELWhich gardener’s tree grew the most in the past year?
52 in.
2011 20122011 2012
72 in.20 in
.
47 in.
61 in.14
in.
Gardener A
Gardener B
www.ksaplus.com
23
VALUE-ADDED MODELWhich gardener’s tree grew the most in the past year when accounting for conditions such as rainfall and temperature?
52 in.
72 in.
47 in.
61 in.+22 in.
+18 in.
Gardener A
Gardener B
2011 20122011 2012
www.ksaplus.com
24
2. Explain the WhyThe Gates Foundation’s MET Project says:
Source: Learning about Teaching, Bill & Melinda Gates Foundation, December 2010
www.ksaplus.com
25
Don’t Make the Perfect the Enemy of GoodTNTP says:
A teacher’s value-added score is comparable to many widely accepted measures of performance in other professions. In order from most stable to least:
Baseball Pitchers: Earned run averages, by year
Insurance Salespeople: Value of policies sold, by month
Baseball Hitters: Batting average, by year
Elementary Teachers: Value-added student growth, by year
Middle School Teachers: Value-added student growth, by year
University Faculty: Student ratings, by semester
Securities Analysts: Commissions, by quarter
Sources: McCaffrey and Sass, Florida State University, and The RAND Corporation, 2009
www.ksaplus.com
26
Comparable MeasuresBrown Center/Brookings says:
SAT/ACT correlations with college success
Mortality rates for hospitals and surgeons
Volume of home sales for realtors
Returns on investment funds
Output of sewing machine operators
Productivity of utilities’ field-service workers
Source: Brown Center on Education Policy at Brookings
www.ksaplus.com
27
3. Explain the HowSix key steps:
1. Select a measure of student achievement
2. Collect individual achievement scores
3. Determine individual growth
4. Select the external factors
5. Examine the effect of various external factors
6. Calculate individual student growth relative to their comparison group
www.ksaplus.com
28
For Example:
www.ksaplus.com
29
4. Anticipate and Combat Resistance
www.ksaplus.com
30
Reminder: It’s about the Kids
“Framing the problem in terms of false negatives places the focus almost entirely on the interests of the individual who is being evaluated rather than the students who are being served.”
—Brown Center on Education Policy at Brookings
www.ksaplus.com
31
5. Be Prepared for Questions
Did my child make a year’s worth of progress in a year? Is my child making progress
toward state standards?
What percent of a teacher’s evaluation will be based on value-added
scores?How will we evaluate
teachers who don’t have them?
How can I be creative if student progress is based on test scores?
www.ksaplus.com
5. Be Prepared for Questions
Technical questions:
Are we able to connect teachers to student test scores?
Is it possible to show progress with all groups of students?
Who will design the value-added model?
Design questions:
What other measures (observation, portfolios, etc.) will be used to evaluate teachers in concert with value-added scores?
How will this affect multiyear tenure (for instance, two-year tenure) if the accuracy of value-added scores improves with three years’ worth of data?
www.ksaplus.com
33
Useful Resources
Communicating about Evaluations Toolkit
http://www.cgcs.org/Page/265
Myths and Facts about Value-Added Analysis — TNTP
http://tntp.org/assets/documents/TNTP_FactSheet_ValueAdded_2011.
Achieve NJ web site for teachers
http://www.state.nj.us/education/AchieveNJ/teacher/
overview.shtml
www.ksaplus.com
34
Final Thoughts
What were your key takeaways from today?
How will you use what we discussed in your state?
www.ksaplus.com
35
Questions and Discussion
www.ksaplus.com
36
Final Thoughts
What were your key takeaways from today?
How will you use what we discussed in your state?
