Farewell to MOOCs: Now Can Real Innovation Begin?

Karen Vignare

Associate Provost

UMUC

Defining MOOCs

MOOC poster April 4, 2013 by Mathieu Plourde licensed CC-BY on Flickrhttp://www.flickr.com/photos/mathplourde/8620174342/sizes/l/in/photostream/

Less Media Hype Better Decisions?

Why do the MOOC? Experimentation Brand Awareness Remediation Move into Online Learning Move to Informal Learning

Less Media Hype Better Decisions…

Why do the MOOC? (cont’d) Access to new students (marketing?) Change role of faculty Change intellectual property rights/ownership Collaboration externally Others?

Mixed Results

• Completion Rates• Surveys of learners• Big Populations• Who Benefits

There is no REAL business model

Venture Capital and others are throwing money at potential

Remember the OL market shifted pretty quickly within 10 years, for-profits owned 50% market share of OL

So now what is the model—same as most internet startups—eyeballs then monetize

Innovation that Matters

• Innovation should be mission driven• Innovation is iterative—not

brainstorming

Push Innovation Throughout

UMUC

Individuals

CIL

To bring the next generation of learning into focus...

To drive the design of innovation in our curriculum, faculty development model, and student support

UMUC Launches the Center for Innovation in Learning

A group of individuals working at the intersection of…

The Undergraduate School The Graduate School

Information TechnologyInstructional Services and

Support

Other Support Units

Scaling, Evaluating, and Improving

• Faculty, staff and departments have been innovating for years

• Most of it is based on hypothesis• Changing times require systemizing• DON’T get rid of what works—measure

it• DO systematize—Innovation &

Collaboration

Center for Innovation in Learning

• Pilots are based on promising research, e.g., student success, efficiency, engagement, openness, learning design

• Pilots must push UMUC to next level• Pilots are internally collaborative• Pilots are both customer facing and

back-end integrated

Current Activities

• Re-envisioned Onboarding (Jumpstart)• Analytics (especially interventions)• OERs/Eresources• Adaptive Learning• Competency Based Education

Jumpstart Grant Synopsis

• To create a personalized learning plan for a student’s first academic year – assessment of skills and personality

variables related to success in higher education and

– evaluation and clarification of life, work, and education goals.

• Serve 5000 Year 1 and 7500 Year 2• Year 3 envisions serving all new students

Course Overview

Welcome

Evaluation

Student Groups

Test• Invited and Enrolled in CAPL 101 (n=40)• Successful Completers (n=31)

Control• Invited and Not Enrolled (n=27)• Random Sample (n=125)

First term new students Fall 2013

Outcome Measures• Semester GPA• Successful course completion • Re-enrollment in a subsequent term

Key Findings

• Students successfully completing JumpStart had significantly higher GPAs, rates of successful course completions, and rates of re-enrollment than the control group.

• Compared to a random sample of students from Fall 2013, all students who participated in CAPL 101 had a significantly higher successful course completion rate.

• Compared to a random sample of students from Fall 2013, students participating in CAPL 101 re-enrolled at a significantly higher rate.

• Students successfully completing JumpStart had significantly higher GPAs, successful course completion rates, and rates of re-enrollment than the random sample of students from Fall 2013

• Continue to pilot and measure various student segments.

Analytics

• Large investments into people, vendors/partners (Civitas/PAR), technology

• Much more difficult is interventions• Cross-functional approach for four

terms with evaluation—but it is just starting

Academic vs. Advising Interventions

Academic• Email only• Includes subject or course-

specific content• Promotes value of course—

what they will learn and why its important

• Highlights key milestones or challenges in the course

• Provides specific tips for success

Advising• Email followed-up by a

phone call• Generic talking points used

for all courses, such as…• Are they prepared and in

the right class?• Do they have any issues

and are they getting the help they need?

• Do they have a clear plan for success?

23

Comments from Students

Thank you for taking the time out of your schedule to welcome me to the program. I appreciate the tips and strategy you have provided and I will try to take full advantage of them.

I am sincerely impressed with the feedback and concern that UMUC displays towards its students.  Thank you again for the encouragement and advice.Thank  you very much... I

appreciate your words of wisdom and I look forward to participating in the program this fall!

Iterations Required

• There is not enough evidence from Spring and Summer to indicate that the treatment is effective.

• Fall results look promising for most UG courses.

Recommendations:

1. Based on positive Fall 2013 results do A/B testing.

2. Continue to refine the research methodology in future pilots.

25

Primary Objective

Determine whether Advising and/or Academic interventions from Fall '13 are most effective and should be operationalized going forward.

Test Strategy

26

A – TestAcademic

interventions

B – TestAdvising

interventions

C – ControlNo

interventions

• Commitment to No Cost for Students

• Almost 300 courses since fall

Adaptive Learning

• Participated and continuing work with Open Learning Initiative

• Investigation of current highly rated tools

• Following work of PLN supported by BMGF

• Considerations of resources and effort• Alignment of courses and pilots• Will initiate for fall

Competency Based Learning

• Outcomes, not inputs• Demonstration of learning• Progress through mastery• Ongoing assessment and support

• Evidence based- Clear measures

- Documentation of mastery

- Early and continuous assessment

• Learner focused- Variable pace and delivery formats- Integration of theory and practice

Value Added

• From seat time to mastery• From assignments to assessments • From one-size-fits-all cohorts to self-paced

progress• From class variance to consistent expectations and

measures• From lecturing to coaching

What Needs to Change?

• Assessment within curriculum• Project based learning • Competency assessments with self-study • Other formats • Varied credit alignment

What Does It Look Like?

Take-aways

• Need internal system-wide innovation to spur entrepreneurialism

• Push individuals to innovate toward top priorities

• Think about people—need to be open, collaborative, and interested in measured change

Questions

Contact Information: Karen Vignare

karen.vignare@umuc.edu