Download - Multiple Measures Assessment Project
Multiple Measures Assessment Project
Welcome WebinarSpring 2014
• Multiple Measures and Common Assessment• Overview of analysis • Student Transcript Enhanced Placement Study (STEPS)
findings leading to Multiple Measures Project (MMAP)• Working with K-12 partners• Compass or Accuplacer data• Role of the pilot colleges • Projected timeline • Questions and Answers
Agenda
Multiple Measures and Common Assessment Initiative• Role of the Common Assessment Advisory
Group
Overview of Analysis• Examine the value of using transcripts as part of the
assessment process• Create predictive models using intersegmental data
to study students who had already taken community college courses
• Use models to analyze how well transcript data predicts the first English & math courses students take and how well they do in them
• Recruit colleges to do local analyses to generate campus-specific insights and trigger CCC/K-12 conversations
4Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
Student Transcript Enhanced Placement Study (STEPS) –
Statewide Findings
In English tests predict tests, grades and courses matter but vary by college
6
7
Cox & Snell pseudo R-square ~ 0.35
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Level
Weakest
Intermediate
Strongest
In math tests predict tests,but high school level is also important
8
9
Cox & Snell pseudo R-square ~ 0.50
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Level
Weakest
Intermediate
Strongest
In English, grades predict grades
10
11
Cox & Snell pseudo R-square ~ 0.20
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Success
Weakest
Intermediate
Strongest
In math, success predictors vary by college
12
13
Cox & Snell pseudo R-square ~ 0.20
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Success
Weakest
Intermediate
Strongest
What Does This Mean?
• Using high school transcript data could help inform alignment/articulation and refine assessment policies
• The relative weight of variables might be influenced by local factors, due to considerations such as articulation and curriculum
• Barriers to implementation must be addressed such as processing transcripts
14Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
What Happens Next?• Updating the statewide analysis• Encouraging more colleges to replicate the study so
they understand the value of transcript data for their own students
• Starting conversations about how the study relates to other aspects of developmental education reform—such as alignment, course offerings, and curriculum
• Helping to develop a tool that enables colleges to access high school data to inform multiple measures assessment
15Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
Working with K-12 partners to upload data• Key aspect of analysis using high school transcript data is
getting local K-12 data into Cal-PASS Plus
• Cal-PASS Plus can assist in outreach, though initially coming from a college helps
• Cal-PASS Plus and RP Group can help provide materials and templates
• Entire process to submit data is streamlined – drag and drop files they already create for the CA Dept. of Education
Compass or Accuplacer data- Does your college use Accuplacer or Compass
for student placements?- If not, we ask that you submit you college’s data
to Cal-PASS Plus to include in the analysis.
Projected timeline Jan – Mar: Build data warehouse for application to multiple measures analysis – collect various data sourcesMay – July: Complete research on system wide multiple measures models and algorithmsJuly – Sept: Develop and deploy online analytic tools and decision tree modelsOct – Dec: Provide PD and direct support to pilot colleges to test multiple measures tools and application; develop and test local user interface to re-identify data for placement decisionsDec – Ongoing: Provide cohort tracking data to pilot colleges on course outcomes; integrate multiple measures project with Common Assessment Initiative
Role of the pilot colleges • Test interactive online placement tools• Provide feedback on user experience• Compare student placements using MMAP analysis to
your college’s current system– How are students placing when comparing the two?– What new conversations have occurred about improving
the placement process?• Work with Cal-PASS Plus to track cohort outcomes
after applying multiple measures through the project
Questions?
Contacts:Mallory NewellThe RP [email protected]
Ken Sorey,Cal-PASS [email protected]
Amanda AvalloneCal-PASS [email protected]
STEPS Report and Summaryhttp://www.rpgroup.org/projects/steps
MMAP Project Sitehttp://www.rpgroup.org/projects/multiple-measures-assessment-project
Cal-PASS Plushttps://www.calpassplus.org/