a310-g inquire: digging into student data wednesday january 12, 2011

A310-G

INQUIRE: Digging into Student Data

Wednesday January 12, 2011

Digging out of snow!

Plus/Delta Protocol

Learning Objective

Understand how digging into multiple forms of data informs our understanding of student

learning.

Agenda for Today

Educational Question

Elluminate Intro

Priority Question

Item Analysis

Student Work

Reflection

BreakWe are here!

And now over to you, Anne…

Testing Audio

1. Click on the Tools menu

2. Select Audio

3. Select Audio Set-up Wizard

4. Follow the on-screen directions. Be sure to select the appropriate Audio Output Device

Change to Wide-Layout View

• Click on View

• Select Layouts

• Select Wide Layout

Testing Chat

• Type your favorite animal into the Chat window

• Send to This Room – press return

Emoticons & Raise Hand

• First, raise your (virtual) hand.

• Second, click on the emoticon that represents how are you feeling about Live-Online so far.

Polling

• How would you rate your stress level for this course right now?

A. High

B. Medium

C. Low

Talking on Microphone

• “Whip Around” – Go down the participant list.

• Click mic icon once to talk – THEN CLICK OFF.

• Whip Around

– Your name

Live-Online Norms

• Participate!

– Emoticons

– Polling

– Chat

– Mic

Educational Question

EDUCATIONAL QUESTION

An Educational Question is…

• Designed to narrow AND be generative– By subject area

• Focused on teaching and learning

• Based on analysis of data– State assessment

• Made by the Data Team

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

Emoticons & Chat-In

• Principal Snow has asked you for feedback on her Educational Question.

– How many students are failing?

– How can we improve student engagement?

– What does the performance of our bubble kids look like?

Data Overview: Intentional Collaboration

• Grouping– Small, mixed (grade/subject) table groups– Data team members facilitate table groups

• What do you see? – One data display at a time

• What do you make of it? – Considering all displays

Patterns/TrendsSimilaritiesDifferencesStrengths Struggles

Getting to a Priority Question

PRIORITY QUESTION

EDUCATIONAL QUESTION

Getting to a Priority Question

• Explain Priority Question– What is a Priority Question?– How does a Priority Question sound?

• Frontload Influencing Factors – Data– Resources– Interest/Buy-in

• Introduce Proposal Protocol

Getting to a Priority Question

• Explain Priority Question– What is a Priority Question?– How does a Priority Question sound?

• Frontload Influencing Factors – Data– Resources– Interest/Buy-in

• Introduce Proposal Protocol

A Priority Question is…

• Designed to help us narrow our focus

• Based on collaborative analysis of data presented in the Data Overview

• Focused on teaching and learning

• Narrowed by subgroup or standard

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

0%10%20%30%40%50%60%70%80%90%

100%

Percentage of Students Scoring Proficient/Advanced by Subgroup

subgroups not mutually exclusiveGrade 10 State Comprehensive Assessment: ELA

Franklin High School, 2010

Pe

rce

nta

ge o

f St

ud

en

ts

Getting to a Priority Question• Priority Question

– How are our students with less than 90% attendance performing on the ELA State Assessment?

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

Getting to a Priority Question• Priority Question

– How are our students with less than 90% attendance performing on the ELA State Assessment?

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

– Examine data by standard for select students to maintain focus on teaching and learning.

Fiction

(8 items)

Dramatic Liter-ature (3

items)

Understanding a Text (1

item)

Style and Lan-guage

(4 items)

Nonfiction

(13 items)

Theme

(1 item)

Structures and Origins of

Modern English

(1 item)

Vocabulary and Concept Devel-opment

(5 items)

Genre

(1 item)

Poetry

(3 items)

0%

20%

40%

60%

80%

100%

Average Percentage of Correct Answers for Each Standard Tested

Grade 10 State Comprehensive Assessment: ELAFranklin High School, 2010 (n=699)

ELA State Standard Tested

Ave

rage

Per

cent

age

of C

orre

ct A

nsw

ers

Getting to a Priority Question• Priority Question

– How are our students performing on the Poetry Standard of the ELA State Assessment?

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

Getting to a Priority Question• Priority Question

– How are our students performing on the Poetry Standard of the ELA State Assessment?

Educational Question

Priority Question

Learner Centered Problem

Problem of Practice

Action Plan

– Examine subgroup data for select standard to narrow your focus further.

Getting to a Priority Question

• Explain Priority Question– What is the purpose of a Priority Question?– How does a Priority Question sound?

• Frontload Influencing Factors – Data– Resources– Interest/Buy-in

• Introduce Proposal Protocol

Frontload Influencing Factors• Data

– What did we learn from our analysis of the data presented in the Data Overview?

• Resources– What knowledge and skills do people in our school and district

already possess?– What curriculum resources can we leverage?– What ongoing professional development can we leverage?

• Interest/Buy-in– Where is the interest, excitement, or momentum?

Emoticons & Chat-In

• How do feel about these factors?

– Data

– Resources

– Buy-In

Getting to a Priority Question

• Explain Priority Question– What is the purpose of a Priority Question?– How does a Priority Question sound?

• Frontload Influencing Factors – Data– Resources– Interest/Buy-in

• Introduce Proposal Protocol

Purpose of Proposal Protocol

• Identify a Priority Question– By standard and/or subgroup

• Structure the Conversation– Maintain focus on making a decision– Think deeply about the rationale

Proposal Protocol

• Guidelines for Proposal– “I propose that we …”

– State one specific proposal at a time (one standard)

– Describe your rationale (influencing factors)

• Protocol– Someone states one specific proposal and rationale.

– The facilitator invites others to add additional rationale in support of this proposal or to add concerns about this proposal.

– The facilitator decides when to take a second proposal

Vocabulary and Concept Devel-opment

5

Structure and Origins of Modern English

1Understanding a Text

1

Genre1

Theme1

Fiction8

Nonfiction13

Poetry3

Style and Language4

Dramatic Literature3

Number of Test Items per Standard

State Comprehensive Assessment: ELA2010

Fiction

(8 items)

Dramatic Liter-ature (3

items)

Understanding a Text (1

item)

Style and Lan-guage

(4 items)

Nonfiction

(13 items)

Theme

(1 item)

Structures and Origins of

Modern English

(1 item)

Vocabulary and Concept Devel-opment

(5 items)

Genre

(1 item)

Poetry

(3 items)

0%

20%

40%

60%

80%

100%

Average Percentage of Correct Answers for Each Standard Tested

Grade 10 State Comprehensive Assessment: ELAFranklin High School, 2010 (n=699)

ELA State Standard Tested

Ave

rage

Per

cent

age

of C

orre

ct A

nsw

ers

Proposal Protocol

• Guidelines for Proposal– “I propose that we …”

– State one specific proposal at a time (one standard)

– Describe your rationale (influencing factors)

• Protocol– Someone states one specific proposal and rationale.

– The facilitator invites others to add additional rationale in support of this proposal or to add concerns about this proposal.

– The facilitator decides when to take a second proposal

From Proposal to Priority Question

• We have discussed three proposals in our small group. How do we get to one?

– There seems to be momentum towards one. Does this sound like something we can work on?

– There is no clear favorite. Let’s vote.

• Thoughts on Consensus

From Proposal to Priority Question

• We now have 6 Priority Questions because we did that activity with 6 small groups!

– Data Team collects, reviews, proposes one to faculty, gets feedback from faculty

– Each group follows their own Priority Question

Questions & Answers (or Break)5 Minutes

Educational QuestionPriority Question

Franklin High’s Priority Question

• How are students performing on nonfiction items?– Rationale for Nonfiction: Standard 13• Data

– 1/3 of items assessed nonfiction standard: 13 of 40

– Nonfiction performance in middle; still needed work

• Resources– Nonfiction performance in middle; some success/knowledge

• Interest/Buy-in– Lends itself to “rigor” more easily than vocabulary standard

– All subjects use nonfiction texts in their curriculum

Digging Into Data

PRIORITY QUESTION

EDUCATIONAL QUESTION

Priority Question: How are students performing on nonfiction items?

Digging Into Data: Item Analysis

• Build Shared Meaning of Standard

• Examine Select Item-Level Data

• Examine Select Items

Nonfiction: State Standard 13

• 10th Grade Nonfiction State Standards

– “Analyze the logic and use of evidence in an author’s argument.”

– “Analyze and explain the structure and elements of nonfiction works.”

Shared Meaning for Nonfiction

• What does the term analyze mean?

– “Analyze the logic and use of evidence in an author’s argument.”

– “Analyze and explain the structure and elements of nonfiction works.”

The New Taxonomy of Educational Objectivesby Marzano & Kendall (2007)

• “Systems of Thinking”

– “Level 6: Self-System”

– “Level 5: Metacognitive System”

– “Level 4: Knowledge Utilization”

– “Level 3: Analysis”

– “Level 2: Comprehension”

– “Level 1: Retrieval”

“Cognitive System”

Analyze: Marzano & Kendall (2007)

• Is the information reasonable?

• What generalizations can I make?

• What might happen next?

• How can the information be organized?

• What are the similarities and differences?

Analyze

• Think of and write down one analysis task students could engage in? (1m)

• All-Chat: Share your idea via chat. (3m)

Shared Meaning for Nonfiction

• What does this content mean?

– “Analyze the logic and use of evidence in an author’s argument.”

– “Analyze and explain the structure and elements of nonfiction works.”

Shared Meaning for Nonfiction

• Individually Review General Standard 13 (4m)

– Circle evidence of what 10th grade standards mean

• “Analyze the logic and use of evidence in an author’s argument.”

• “Analyze and explain the structure and elements of nonfiction works.”

Shared Meaning for Nonfiction

• Whole Group (4m)

– What is your evidence? Raise hand to get on mic

• “Analyze the logic and use of evidence in an author’s argument.”

• “Analyze and explain the structure and elements of nonfiction works.”

Data for Nonfiction

• Individually Review Item-Level Data (4m)– Stay at the level of the data

• Whole Group– Chat-In: What do you see? (5m)

– On Mic: How might you group these items? (5m)• We group to focus and save time

Franklin High’s Priority Group

ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL

1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%

29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%

Franklin High’s Priority Group

ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL

1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%

29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%

Franklin High’s Priority Group

ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL

1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%

29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%

Items for Nonfiction

• Individually: Time to read the passage. (5m)

• Examine Item 5– Answer the question yourself. (15s)– Chat-In: What does a student need to know and

be able to do to answer this question correctly? (5m)

– On Mic: What are you theories about why students choose each incorrect answer. (5m)

– *Repeat for Items 6 & 7

Learning from Item Analysis

• *How can we describe the strengths and struggles of our learners?

– Free write

– Chart paper and markers

– Think, pair, share

Chat-In & Mic

• Imagine yourself “on the balcony…”

– What adaptive challenges might we face when

moving from an Educational Question to a Priority

Question to an Item Analysis?

Digging Into Data

PRIORITY QUESTION

EDUCATIONAL QUESTION

Priority Question: How are students performing on nonfiction items?

Looking at Student Work

• *Individually: Complete the prompt (4m)

• *Triads: Share your approach (6m)

• Individually: Review student work (5m)– Stay at the level of the data

Looking at Student Work Protocol

• On Mic: Whole Group

– What do you see? (10m)

• Starting with Student 1

– What do you make of it? (10m)

• Trends & patterns

• Similarities & differences

• Strengths & struggles

Priority Question: How are students performing on nonfiction items?

Digging Into Data

PRIORITY QUESTION

EDUCATIONAL QUESTION

Priority Question: How are students performing on nonfiction items?

LEARNER CENTERED PROBLEM

A Learner-Centered Problem is…

• … a statement about student learning, not a question.

• … based on evidence we found when digging into data.

• … directly related to your Priority Question.

• … specific and small.

ExamplesPriority Question Learner Centered Problem

Writing: How do students use evidence from text to support their argument?

Students don’t explain how quotes support their argument.

Reading: What strategies do students use for comprehension?

Students don’t look for connections to their own lives as they read.

Math: How do students approach word problems?

Students translate word problems into equations and solve without thinking about what the problem is asking.

Reflection

Taking Stock of your Learning

• Revisit what you wrote yesterday about maintaining a relentless focus on Evidence in all conversations

3 minutes

Plus/Delta Protocol

See you tomorrow…