dec. 11, 2012, sung- youn choi
Post on 23-Feb-2016
40 Views
Preview:
DESCRIPTION
TRANSCRIPT
Students Learning in the IQWST & LP
Improving middle school students’ understanding of core science
ideas using coherent curriculum
Dec. 11, 2012, Sung-Youn ChoiPost-doc of CREATE 4 STEM Institute, Michigan State University
IQWST & LP
What will we do today?
• Examine why we should redesign curriculum materials• Introduce the IQWST curriculum design; the next generation of
middle school science materials• Introduce the test design; aligned with a LP approach• Discuss key design principles and the CCD process• Share some of our findings; using IRT analysis• Discuss about improving middle school students’ understanding of
core science ideas using coherent curriculum
– Investigating and Questioning our World through Science and Technology (IQWST)
– Learning Progression (LP) – Construct-Centered Design Process (CCD) – Item Response theory (IRT)
2
IQWST & LP 3
Why redesign curriculum?
• Inadequate Science Materials– Cover many topics at a superficial level– Focus on technical vocabulary– Fail to consider students’ prior knowledge – Lack coherent explanations of real-world phenomena – Provide students with few opportunities to develop explanations of
phenomena– Materials lack coherence and as a result don’t focus on building
understanding over time• Students need to be prepare for the world in which they live now
and will live as adults– Deeper understanding of the core ideas becomes instrumental for
competitiveness in a global economy, as well as for the personal well being of each citizen
– Students need deeper knowledge in order to use understanding to solve problems
IQWST & LP 4
Design the Next Generation of Science Learning Environments
• Solution• The Investigating and Questioning our World through Science and
Technology (IQWST)– Utilizing a coherent approach for building curriculum materials across the
school years– Using learning goals as a focus– Supporting students in developing understandings of the core ideas of science
and scientific practices
– Applying what we know about student learning– Promoting literacy practices – Engaging students in complex tasks – Using a project-based approach
IQWST & LP 5
IQWSTDesign Principles
IQWST & LP
Key Design Principles
• Curriculum Coherence
A. Inter-unit coherence• Build links across units among the grades or content areas• Provide opportunities to develop, reinforce, and use their understanding
during their three-year IQWST experiences
B. Intra-unit coherence• Use driving-questions as support to link ideas together • Create alignment by iteratively aligning learning goals with tasks
C. Learning-goals coherence• Use big ideas• Unpack standards from a learning perspective• Create learning performances as a way to specify knowledge in use
6
IQWST & LP 7
A. Inter-unit Coherence: Structure of IQWST• The IQWST is built on scientific core ideas.
Physics Chemistry Biology Earth Science
6th gradeLight & its
interaction with matter
Particle nature of matter, phase
change
Survival: From organisms to ecosystems
Water & rock cycles
7th gradeConservation & transformation
of energy
Nature of chemical reactions
Biological organization & development:
Cells to systems
Surface & atmospheric
processes behind weather
and climate
8th grade Laws of motionPhotosynthesis and respiration(carbon cycling)
Heredity, Natural selection
Large-scale geological
processes on earth & other
planets
Units
IQWST & LP 8
A. Inter-unit Coherence: Scope and Sequence• The core ideas are coordinated across the grades and the contents.
Physics Chemistry Biology Earth Science
6th gradeLight & its
interaction with matter
Particle nature of matter, phase
change
Survival: From organisms to ecosystems
Water & rock cycles
7th gradeConservation & transformation
of energy
Nature of chemical reactions
Biological organization & development:
Cells to systems
Surface & atmospheric
processes behind weather
and climate
8th grade Laws of motionPhotosynthesis and respiration(carbon cycling)
Heredity, Natural selection
Large-scale geological
processes on earth & other
planets
IQWST & LP 9
B. Intra-Unit Coherence: Driving Questions
• Driving Question: – Links content in a context– Links tasks, investigations, and assessments to learning goals– Ties the unit together– Builds intra-unit coherence– Uses rich and open-ended questions with everyday language
IQWST & LP 10
C. Learning Goals Coherence
• Developing Learning Goals• The construct-center design
approach (CCD)– Step 1: Define the constructs
• Correcting ideas and concepts to learn
• Identifying potential students difficulties and alternative ideas
– Step 2: Unpack the ideas and concepts
• Creating a set of claims• Specifying evidences
– Step 3: Design tasks and materials
– Step 4: Review Products
IQWST & LP
IQWSTExample lesson
11
IQWST & LP 12
IQWST Smell Unit
• 8-week, project-based unit for 6th grade students• Driving Question: How can I Smell Things from a Distance?
• Three Learning Sets, 15 Lessons– Learning Set 1: Students construct models to help them understand the
particle nature of matter while focusing on the behavior of gases
– Learning Set 2: How models help to explain why different materials have different properties
– Learning Set 3: Using the particle nature of matter model to explain phase changes
Ideas from all three learning sets are brought together through a culminating final task in which students use their knowledge of the particle nature of matter.
IQWST & LP 13
Driving question for UnitScientific Core Idea
Particle Nature of Matter
Properties of Matter
Phase Changes
IQWST & LP 14
Example (1) activity
• Learning Goal: Lesson 13 – Students will explain phase changes from gas to liquid to solid and from solid
to liquid gas at the molecular level.• Scientific Principle
– Evaporation
IQWST & LP 15
Driving question
Example (2) reading
• Scientific Principle– Condensation
• Alternative Conceptions– Condensation on the outside of a container is water that seeped through the container
itself (or sweated through the walls of the container).– The coldness comes through the container and produces water.– Condensation is when air turns into a liquid.
Prior knowledge
Everyday experiences
Question & writing opportunity
Set purpose for reading
IQWST & LP
Example test items
16
IQWST & LP 17
Example item (#1) PC02
• Contents area– Phase change, Structure
• Claim– Students can explain how condensation occurs (without a particulate model).
• Unpacking/evidence– Level 1. Matter can exist as a solid, liquid or gas. Generally, the same pure
substance can exist in all three states. The state of matter depends on the temperature.
IQWST & LP 18
Example item (#1) PC02
• Contents area– Phase change, Structure
• Claim– Students can explain how condensation occurs (without a particulate model).
• Unpacking/evidence– Level 1. Matter can exist as a solid, liquid or gas. Generally, the same pure
substance can exist in all three states. The state of matter depends on the temperature.
PC02
Gas Liquidd = 0.958 (hard)
18.4% 29.0%N1=1092, N2=399
IQWST & LP 19
Example item (#2) PC18
• Contents area– Phase change, Conservation
• Claim– Students can explain the mass of matter are conserved in transforming from
solid to liquid or solid to gas (without a particulate model).• Unpacking/evidence
– Level 1. Matter is not created or destroyed when substances change form or change into other substances (e.g., crushing, dissolving, phase change, chemical reaction).
IQWST & LP 20
Example item (#2) PC18
• Contents area– Phase change, Conservation
• Claim– Students can explain the mass of matter are conserved in transforming from
solid to liquid or solid to gas (without a particulate model).• Unpacking/evidence
– Level 1. Matter is not created or destroyed when substances change form or change into other substances (e.g., crushing, dissolving, phase change, chemical reaction).
PC18
Solid Gasd = - 0.253
23.6% 34.9%N1=368, N2=175
IQWST & LP 21
Example item (#3) PC19
• Contents area– Phase change, Conservation
• Claim– Students can explain how condensation occurs (without a particulate model).
• Unpacking/evidence– Level 1. Matter is not created or destroyed when substances change form or
change into other substances (e.g., crushing, dissolving, phase change, chemical reaction).
PC19
Solid Liquidd = - 0.780 (easy)
35.8% 41.9%N1= 358, N2=201
IQWST & LP 22
Example item PC02, PC18, PC19
• Can we give same credit to each item?• Do students need to take every item?
PC02 PC18 PC19
Phenomena Phase Change
Contents area Structure Conservation Conservation
Concept Gas Liquid Solid Gas Solid LiquidItem difficulty d = 0.958 (hard) d = - 0.253 d = - 0.780 (easy)
% of correct answer(pretest) (posttest)
18.4% 29.0% 23.6% 34.9% 35.8% 41.9%
# of students(pretest), (posttest)
N1=1092, N2=399 N1=368, N2=175 N1= 358, N2=201
IQWST & LP
Item Response Theory: IRT analysis vs. CTT (classical test theory)
Creating test forms
23
IQWST & LP 24
A. Scoring system
• CTT (raw score)– O X O O O = 4 (total score)– How many correct answers did you get? – The original result obtained by a student on a test. (% of correct answer, mean
score, total score, or etc.)
• IRT (latent ability score/logit)– ? ? ? ? ? = 1.3 – What is the probability of a correct response to an item?– Using the IRT model (latent trait models), we can estimate overall latent ability
score/ logit. – In three parameter logistic model (3PL)
• difficulty• discrimination• pseudo-guessing
IQWST & LP 25
Example: PC02 (ice cup, condense)
• The item difficulty is determined by the latent trait logit of a person who has 50% of right answer. In this graph, 0.5 probability meets the trait graph at 0.958.
• The MNSQ fit is around 1. It has good fitness between observed responses and latent trait model.
Item difficulty = 0.958
Observed response
Latent trait model
IQWST & LP 26
Example: PC18 (solid gas)
• The item difficulty is determined by the latent trait logit of a person who has 50% of right answer. In this graph, 0.5 probability meets the trait graph at -0.253.
• MNSQ fit is around 1. It has good fitness between observed responses and latent trait model.
Item difficulty = - 0.253Latent trait model
Observed response
IQWST & LP 27
B. Test form
• CTT – (missing data) O X O O O _ _ = 4, or deleted– The raw score, which is the number of items correct, does not present a broad
picture of test performance because it can be interpreted only in terms of a particular set of test questions.
• IRT – (missing data) ? ? ? ? ? _ _ = 1.7– Linking item– Latent student ability score allows direct comparisons of student performance
between specific sets of test items.
– IRT generally brings greater flexibility and provides more sophisticated information.
IQWST & LP 28
B. Test form (LP test design)
• Learning Progressions– LPs are research-based descriptions of how students build their knowledge
within and across disciplines (topics) over a broad span of time– Explaining many important phenomena requires incorporating ideas from
many topic areas– Focusing on the growth of sets of ideas instead of individual topic
PhaseChange
ChemicalReaction Others
Phenomena
Structure of matter
Conservation
Interactions
Topic areas
IQWST & LP 29
• Learning Progressions– LPs are research-based descriptions of how students build their knowledge
within and across disciplines (topics) over a broad span of time– Explaining many important phenomena requires incorporating ideas from
many topic areas– Focusing on the growth of sets of ideas instead of individual topic
PhaseChange
ChemicalReaction Others
Phenomena
Structure of matter
Conservation
Interactions
Topic areas
B. Test form (LP test design)
Level 1
Level 2
Level 3
Level 4
Level 5
Level 6
IQWST & LP 30
B. Test form (LP test design)
• Test form design– Total 62 items; 52 multiple-choice items and 10 open-ended or short-answer
items
• A-version Form (1A, 2A, 3A 4A)– Each containing 14 items– For 6th, 7th grade
• B-version From (1B, 2B, 3B, 4B)– Each containing 15 items – For 8th grade– Three items were used to adjust the difficulty between A- and B-version
• Linking items– Each test form contained three pairs of linking items– Total six linking items were contained– Linking items cover a range of contents areas along levels 1 to 3 on the LP
IQWST & LP
Research Questions &Findings
31
IQWST & LP 32
Research Questions
• Do the chemistry IQWST units, which link content within and across units, build understanding of targeted chemistry ideas?
– Comparing students’ performances and learning gains in Phase Change and Chemical Reaction
– Comparing Students performances among Topic Areas
• Do student apply what they learned from IQWST Units to solve new problems?
– Comparing between IQWST items and non-IQWST items in Chemical Reaction
IQWST & LP 33
Data Collection: pre- and post-test
• Participants– Total 1,519 students
• 481 students in 6th grade, 519 in 7th grade, 519 in 8th grade– 7 teachers– 6 schools
• Two of the schools represent middle- to upper-middle class socioeconomic status
• Four of the schools were from neighborhoods that are lower-SES, with a majority of students eligible for free and reduced lunch.
• Research period– Schools used IQWST materials during national field trials (2007~2010)– Pretest (fall, 2010), Posttest (Spring, 2011)
IQWST & LP 34
Test Instrument
• Hypothetical LP based test– We developed test instruments based on a Learning Progression (LP) for the
transformation of matter– The assessment instruments measured how students applied ideas within and
across topics to explain phenomena involving transformations of matter.
• Test forms– total 62 items, 8 different forms, each form contained 3 pairs of linking items for a
total of 6 linking items.
• Validation of instrument– Piloted with students from grade 6th -15th (N=792). – Content validity: semi-structured interviews with a subset of students
supplemented the pilot test data.– The LP research team is contributing to validate instruments. (Namsoo Shin,
Shawn Stevens)
IQWST & LP 35
Framework (1)
• Phenomenon– Phase Change: 6th grade IQWST unit, 21 test items (20 with IQWST)– Chemical Reaction: 7th grade IQWST unit, 16 test items (11 with IQWST)
PhaseChange
ChemicalReaction Others
Phenomena
Structure of matter
Conservation
Interactions
Topic areas
IQWST & LP 36
Framework (2)
• Topic areas– Conservation– Structure of matter; matter and materials, properties and periodicity, atomic
structure– Interactions; kinetic energy, forces, energy, processes
PhaseChange
ChemicalReaction Others
Phenomena
Structure of matter
Conservation
Interactions
Topic areas
IQWST & LP 37
Item analysis
• The purpose of item analysis is to provide how well assessments work, and how well individual items on assessments work.
• Item Response Theory (IRT, Wu, et al., 2007)– IRT allows us to evaluate students’ ability and to describe how well items on
the test are performing. In IRT, ability and item parameters are both estimated based on students’’ response patterns on the test.
– Item parameters (difficulty, discrimination, pseudo-guessing)were used to determine whether an item displayed sound psychometric properties
• General item analysis criteria• Items difficulty (estimate)• MNSQ fit (the weighted means square) ~ 1• T (pseudo-guessing) <3 • Item correlation > 0.3• % of correct answer (total)
IQWST & LP 38
Analyzing data set
• Phase Change (20 IQWST items + 1 non-IQWST item)– Conservation (4)– Structure of matter (8+ 5 overlapped with interactions + 1 non-IQWST)– Interactions (3+ 5 overlapped with structure)
• Chemical Reaction (11 IQWST items + 5 non-IQWST items)– Conservation (6+ 2 overlapped+ 1 non-IQWST)– Structure of matter (2+ 2 overlapped+ 5 non-IQWST)– Interactions (2+ 1 non-IQWST)
IQWST & LP 39
Result fromPhase Change
IQWST & LP 40
Item analysis (PC)
• Accepted items range • - 0.988 < d <1.174• MNSQ fit <1.17 • - 4.0 < T < 3.0• 0.35 < r
Problematic item
IQWST & LP 41conservation
structure
Interactions
conservation
structure
Interactions
Students distribution Item diff distribution
Item difficulties & Latent distribution (PC)
IQWST & LP 42
Scoring
• Students latent ability logit– The partial credit, multi dimensional model – The expected a posteriori (EAP) parameter was standardized to
present student ability score
• Students ability score– To facilitate the interpretation of their ability scores assigned to
students– Normalization as t2 so that the mean and S.D of scores was 500
and 100
IQWST & LP
• Phase Change; Topic- 1. Conservation
43Distribution of Students
6th gradePretest Posttest
7th gradePretest Posttest
8th gradePretest Posttest
Stud
ents
’ abi
lity
scor
e
R1. Do students develop understanding of Phase Change?
IQWST & LP
• Phase Change; Topic- 1. Conservation
44
% of variance= 0.46 **
% of variance= 0.45 **
% of variance= 0.22
R1. Do students develop understanding of Phase Change?
IQWST curriculum
IQWST & LP
• Phase Change; Topic- 2. Structure of Matter
45Distribution of Students
6th gradePretest Posttest
7th gradePretest Posttest
8th gradePretest Posttest
Stud
ents
’ abi
lity
scor
e
R1. Do students develop understanding of Phase Change?
IQWST & LP
• Phase Change; Topic- 2. Structure of Matter
46
% of variance= 0.63 **
% of variance= 0.45 **
% of variance= 0.28 **
R1. Do students develop understanding of Phase Change?
IQWST & LP
• Phase Change; Topic- 3. Interactions
47Distribution of Students
6th gradePretest Posttest
7th gradePretest Posttest
8th gradePretest Posttest
Stud
ents
’ abi
lity
scor
e
R1. Do students develop understanding of Phase Change?
IQWST & LP
• Phase Change; Topic- 3. Interactions
48
% of variance= 0.48 **
% of variance= 0.49 **
% of variance= 0.37 **
R1. Do students develop understanding of Phase Change?
IQWST & LP 49
Result fromChemical Reaction
IQWST & LP
Item difficulties & Latent distribution: (2) CRconservation
structure
Interactions
conservation
structure
Interactions
Students distribution Item diff distribution
IQWST & LP 51
• IQWST aligned items• Non-IQWST
R2. Do Students apply their learning to solve new problems?
IQWST & LP 52
Students’ development for IQWST items and non-IQWST
% of variance= 0.25 **
% of variance= 0.12 **
IQWST & LP 53
R1. Do students develop understanding of Chemical Reaction? (with IQWST items)
• D1. Conservation
% of variance= 0.62 **
% of variance= 0.40 **
% of variance= - 0.08
IQWST curriculum
IQWST & LP 54
R1. Do students develop understanding of Chemical Reaction? (with IQWST items)
• D2. Structure of Matter
% of variance= 077 **
% of variance= 0.45 **
% of variance= - 0.05
IQWST & LP 55
R1. Do students develop understanding of Chemical Reaction? (with IQWST items)
• D3. Interactions
% of variance= 0.29 **
% of variance= 0.41 **
% of variance= 0.10
IQWST & LP 56
Interpretation of the finding
• Demonstrate that students’ latent ability changes across grade level, supporting the hypothesis that coherent curriculum materials can support students in developing integrated understanding.
• Provide support that curriculum coherence may increase student achievement and build integrated understanding
• Suggest that curriculum coherence can help students develop further understanding then when ideas are not linked
IQWST & LP 57
Acknowledgments
• Thanks to the IQWST and the LP research team • Thanks to the participating schools, teachers, and students• Email me; choisun9@msu.edu
top related