TIPS FOR LEVEL 4 RESPONSES ASSESSMENT AND EVALUATION

Public School Academies Unit

Office of Education Improvement and Innovation

Michigan Department of Education

November 7th, 2012

Agenda of Today’s Webinar

• Welcome and introductions

• “Housekeeping” items

• Audience

– New applicants

– Awardees getting ready to move to Stage 2

• Webinar objectives:

– Participants will be able to articulate the components necessary to score a “4” on questions 18, 19 and 20 of the planning grant application

– Participants will be able to cite examples of concepts, data and text that should be included in their responses to questions 18, 19, and 20

– Participants will be able to navigate to web resources that can help as they prepare their narrative responses

What are peer reviewers looking for? • Proposed assessment of growth and achievement

covers the following:

– All students (e.g. grades being offered)

– All content (e.g. subject matter and delivery mechanism)

– Is ongoing (continuous improvement and innovation)

– Capable of shaping and improving teaching and learning

– Extensive enough to determine whether the educational goals are being achieved.

Understanding Reviewers Comments

General Analysis- Questions 18 and 20

• Rated by reviewers as below average 37% of the time.

• 20 of the 37% is rated as not addressed/weak

General Analysis- Question 19

• Rated by reviewers as below average 37% of the time.

• 20 of the 37% is rated as FAIR

Insight

Peer Reviewer Comments-Assessments

• Lacks specific metrics to align outcomes to instructional strategies

• The project does not articulate an ongoing strategy to monitor and assess student academic achievement with standardized measures, nor a plan for informing students of their progress.

• Lacks a plan for data-driven decision making; execution of the use of data is not evident

• Yearly progress goal is unreasonable. Use of pre and post testing during the same year recommended

Assessments- Continued

• Assessment of student achievement lacks specifics, including all the assessment designs, how often they will be given, and how MI grade-level content expectations will be tied to them

• How will parents receive regular, confidential, reports from standardized testing and progress reporting?

• Does not provide wide variety of assessment tools. There is no discussion of how formative and summative assessments will be used to document student

• There is no discussion of how teachers will use these assessments to modify their teaching

Peer Reviewer Comments for Q 20

• The MI School Improvement Framework strands should be the basis for data collection. No reference to how attendance, behavior, planned participation, teacher attendance data will be collected

• How will the data be used to improve instruction? Lacks specificity and a plan to collect data

• The response misses Leadership, Personnel and Community components in the SIF and does not provide much narrative about what is going to be done with the data and how a feedback loop will occur toward implementing change based upon the data collected.

• This would benefit from a narrative creating linkage between data collection and the school improvement framework.

• Application could benefit from listing specific events and tools for data collection (i.e., parent surveys)

• Assessment Program • Connecting the Educational

Program to goals • Improving teaching and

learning

Question 18

Question 18

Narrative Template

Describe the assessment program and related strategies, detailing how

assessments connect to the educational program and its goals. Provide a thorough

description of how assessment results will be used to improve teaching and learning for all students in all content areas.

Rubric

Proposed assessment of growth and achievement covers all students and content, is ongoing, capable of shaping and improving teaching and learning, and extensive enough to determine whether the educational goals are being

achieved.

Concentrating your focus

• What is your assessment program?

• What are the related strategies of your assessment program?

• How does your assessments connect to the educational program and its goals (i.e. educational goals written from question #8).

• How will your assessment results will be used to improve teaching and learning for all students in all content areas. Provide a thorough description.

Walking through an example

2012 PLAN to ED-Performance 3025

2952

2925

2916

2768

18

2998

2810 2801 2798 2786 2725 2708 2703 2692

2488

17

2942 2896 2818

2791-2 2782 2770 2764 2758 2752

2740-2 2709-2 2694-2 2684

2676-2 2664-2 2633

2588-2 2572 2565 2524

16

2936

2813 2803

2787-2 2763-2 2733 2719

2700-3 2674-2 2649-2 2636-2 2621-2 2607-2 2602 2574

2568-2 2556 2538 2522

15

5 Students on path to ACT 21-

20

10 Students on path to ACT 20-

19

27 on path to ACT 19-

18

29 on path to

ACT 17-18

EP-7th grade

EP-9th grade

EP-9th grade

EP-10th grade

2012 PLAN to ED-Performance 2915

2866

2802-2 2789 2754 2683

2673-2 2645-2 2620-2 2599

2553-2 2530-2 2504-4 2456-2 2401-2 2380

2360-2

2170-2

14

2722 2700 2693

2686

2612-2

2586 2548 2521

2462

2370

13

2832

2782

2652 2644 2629

2595 2571 2520 2502 2493 2418

2339 2335

2174

12

2713

2693 2678 2674 2636 2617 2605 2575 2568 2547 2520 2459 2430 2424 2418

2247 2236 2208 2174

11

31 on path to

ACT 16-15

11 on path to

ACT 15-14

14 on path to

ACT 14-13

19 on path to

ACT 13-12

EP-5th grade

EP-6th grade

EP-5th grade

EP-5th grade

VISION

Students can narrow math gap if given tailored lessons to remediate gaps in earlier years using study island

Provide a comprehensive multi-grade assault on math skills to change outcomes in 11th grade

Utilization of both push and pull incentives will be successful in achieving student effort

Result will be making AYP goals yearly

Framework for Success SUMMER

STUDY ISLAND: ALL GRADE LEVELS SUMMER BOOT CAMP

MATH SCHOOL IMPROVEMENT ACTION COMMITTE

FALL SEMESTER ED PERFORMANCE TESTING

ACHIEVEMENT LEVEL GROUPING IN MATH STUDY ISLAND - WEEKLY DURING CLASS & HOMEWORK

WINTER SEMESTER ACHIEVEMENT LEVEL REGROUPING

ED PERFORMANCE TESTING POST EVALUATION USING NPR GAINS TO ASSESS RESULTS

PHASE 1 (summer 2012 and academic year 2012/2013) of MD

SMART goals initiative will consist of:

PILOT INTEGRATION of 4 COMPONENTS

• EDPERFORMANCE

• STUDYISLAND

• SKYWARD GRADEBOOK

• INSTRUCTIONAL SUPPORTS: (Summer Bridge, After-school tutoring, IEP/Resource teachers, Title I resources, technology integration)

PHASE 2 (Summer 2013-academic year 2013/14) of MD SMART

goals initiative will consist of:

• Continue to develop MD SMART infrastructure and capacity reflecting and revising our emerging model;

• Design summer mathematics program infrastructure (to include: accelerated, bridge, and boot camp)

Expected Ed-Performance Scale Score Growth

8th to 9th

Grade

100 pts

9th to 10th

10th to 11th

90 pts

80 pts

Gain

ORA Longitudinal Growth Model

2012 9th

2013 10th

2014 11th

2015 12th

ED-P

ERFO

RM

AN

CE

NAT

ION

AL

PER

CEN

TILE

S

50th

20th

40th

60th

80th

100th

GAP CLOSING – VALUE ADDED

Communication Plan

• Marketing of frame work

– To parents/guardians

– Students

• Fall parent student meetings

• Acceptance letters

• News letters

• Web site

• Student handbook

Question 19

Narrative Template

Describe the anticipated annual standards or measures of student achievement that you expect. Take into account that MEAP is not given for every grade in every year. How will you set annual growth targets? How will you communicate progress toward the standards/targets to students and parents?

Rubric

Annual standards for student achievement and growth have been established; they reflect the anticipated student population. An effective process for informing parents and students about progress has been provided.

Question 19

• Describe the anticipated annual standards or measures of student achievement that you expect.

• How will you set annual growth targets?

• How will you communicate progress toward the standards/targets to students and parents?

Frame Work Road Map SEMESTER I

CURRICULUM TOPICS SEMESTER II

CURRICULUM TOPICS

TERM 1

TERM 2

TERM 3

TERM 4

TAILORED STUDY ISLAND BASIC SKILLS REMEDIATION

ED PERFORMANCE

BASE LINE Using EP criteria

ED PERFORMANCE

CAPSTONE Using student

growth

TAILORED STUDY ISLAND BASIC SKILLS REMEDIATION

SKYWARD GRADEBOOK

80-90%

80-90%

80-90%

80-90%

ACT TESTS • EXPLORE • PLAN • ACT

Frame Work Road Map SEMESTER I

CURRICULUM TOPICS SEMESTER II

CURRICULUM TOPICS

TERM 1 TERM 2 TERM 3 TERM 4

TAILORED Study-Island SKILLS REMEDIATION • Driven by Ed-Performance base line • Craft grade level lessons per student • 9-11 Math teachers use S-I one or more days each week • Grades transferred to Grade book in a 10% category on weekly basis • Extra credit earned by extra efforts

ED PERFORMANCE BASE LINE

Using EP criteria • Grade earned

for Ed Performance results

• Test outcomes direct Study Island lessons

ED PERFORMANCE CAPSTONE

• Use student growth in grade book

• Min of 50 pts

SKYWARD GRADEBOOK

80-90%

80-90%

80-90%

80-90%

ACT TEST suite • Used to

validate Ed Performance

• Grade given for ACT test results

Gra

de

bo

ok

up

dat

es

Question 20

• Michigan’s School Improvement Framework

• Data collection

Question 20

Narrative Template

Michigan’s School Improvement Framework (SIF) outlines objectives of public schools that go beyond student achievement – for instance, leadership, climate, community involvement, teacher retention etc. (See the full SIF at http://www.michigan.gov/documents/SIF_4-01-05_130701_7.pdf) Identify what data the proposed school will collect to determine whether the school is achieving these objectives.

Rubric The school has a plan to collect data beyond student achievement and to self-assess across the School Improvement Framework categories (Teaching/Learning, Leadership, Personnel & Prof. Dev, School-Community Relations, Data & Info Management

School Improvement Framework

School Improvement Framework Rubrics http://www.michigan.gov/documents/OSI_FW_Rubrics_v_157013_7.3.pdf

Strand Standard Data Collection

Teaching For Learning

Curriculum is Aligned, Monitored and Reviewed

ABC Academy will have detailed and tailored pacing guides, scope and sequence documents (aligned vertically and horizontally) and curriculum maps. Curriculum alignment will be a regular agenda item in weekly mandatory grade group meetings, as minutes will show. Teacher unit plans and lesson plans will show alignment of instruction within grade levels.

Teaching For Learning

Curriculum is communicated

Leadership School leaders are knowledgeable about schools educational program

Research Reminders and Tips

• Spend time researching assessment programs, techniques, and models that can connect to your individual curriculum and instructional plans.

• Usually, you will need to plan on using parts of different models to create your individual assessment plan.

• Will the components of the proposed assessment program work in your environment?

Research Reminders and Tips

• Drill down your research efforts to match your individual proposed school. When searching on-line, include your predicted criterion- urban/rural/suburban school, demographics, grade levels, unique class offerings or focus areas, curric./instruction models, etc. K-12 Assessments will get you over 16,000,000 hits!

Is your proposed plan complete?

• How do you know when you are assessing enough? Will you gather enough individual and group achievement and growth data to make educated decisions, evaluate staff, modify curriculum/instruction, determine if goals have been met, meet all requirements, etc.

Is your proposed plan complete?

• Have you articulated a diverse plan that:

• prepares for formative, interim, and summative assessments

• is set up for all students to be assessed every year

• includes plans for what to do with students who are unsuccessful on assessment

• gives you the metrics and results that match your measurable educational goals

• produces both growth and achievement data for individual students and all student groups

Is your plan realistic?

• Students and teachers need enough time to learn, create, solve, review, practice, interact, discuss, fix, compare, contrast, observe, and on and on. Are you taking that into consideration when you create your assessment program?

• Do not try to impress the reviewers so much that you go overboard!

QUESTIONS?

38

Points of Contact • Beatrice Barajas, Secretary barajasb1@michigan.gov

• Kim Sidel, Analyst sidelk@michigan.gov

• Tammy Hatfield, Consultant hatfieldt@michigan.gov

• Karla Browning, Consultant browningk2@michigan.gov

• Neil Beckwith, Consultant beckwithn@michigan.gov

• Ron Schneider, Consultant schneiderr8@michigan.gov

• Phillip Caldwell II, Manager caldwellp@michigan.gov

• Mark Eitrem, Supervisor eitremm@michigan.gov

(517) 373-4631 MDEPSAGRANT@MICHIGAN.GOV

http://www.michigan.gov/charters