PeerWise: Student-generated multiple-choice questions for

enhanced engagement and learning

Judy Hardy j.hardy@ed.ac.uk

Institute of Physics Higher Education Group Technical Meeting, Edinburgh, October 2013

2

PeerWise: Motivation

3

Time on task

For a course with N contact hours per week, how much time do you spend on self-study? 1. ~0 2. N/2 3. ~N 4. 2N 5. >2N

How much time do think you should spend on self-study? 1. ~0 2. N/2 3. ~N 4. 2N 5. >2N

4

PeerWise

•  Web-based MCQ repository –  Free to use, developed by Paul Denny, University of Auckland

•  2007 – 2010: > 40 institutions > 20,000 students > 50,000 questions > 1,000,000 answers

•  2007 – 2013: > 700 institutions > 100,000 students > 500,000 questions > 10,000,000 answers

5

PeerWise

•  Content built by students: –  Develop questions with associated explanations –  Answer questions written by their peers –  Rate questions for quality and difficulty –  Take part in discussions with question author –  Follow other authors –  All activity is anonymous

6

Example Implementation

•  Physics 1A: PeerWise replaces two weekly handins •  Activity introduced in workshop session

–  Students worked through structured example task and devised own questions in groups

•  Two assessment deadlines –  But available to the end of the course

•  Counts 4% of total course mark •  Minimum requirements

–  Write 2 questions –  Answer 10 questions –  Rate & comment on 6 questions

7

Implementation cont.

•  We are deliberately hands off: –  No moderation –  No corrections –  No interventions

•  But we observe…..

8

Student contributions

•  5 courses from 3 science disciplines at 3 universities –  Physics, Chemistry, Biology / Edinburgh, Glasgow, Nottingham –  ~850 students and ~10 members of staff

9

Student contributions

•  Participation above minimum requirements •  Average quality generally good

–  Problems not ‘bookwork exercises’ –  Few trivial questions / nonsense distracters

•  Comments on questions constructive –  ‘Community moderation’ corrected sub-par questions

•  (Usually) didn’t leave everything to the last minute

10

End of course performance (Physics 1A)

11

PeerWise activity Effect sizea

Number of questions authored 0.24***

Number of questions answered 0.24***

Average comment length (chars) 0.22**

Days of activity 0.33***

aSpearman’s partial correlation coefficient between PeerWise activity and end-of-course exam mark, controlling for pre-course FCI mark **p<0.01 ***p<0.001

End of course performance

12

Learning networks

13

Summary

•  High level of engagement –  Both number and quality of contributions –  Scaffolding is important

•  Positive correlation with end-of-course performance –  Even after taking prior ability into account –  Complex pattern in terms of types of activity –  But don’t underestimate time on task

•  Not specific to discipline or institution

14

For more information

PeerWise http://peerwise.cs.auckland.ac.nz

15

PeerWise Community http://www.peerwise-community.org

Acknowledgements

Thanks to the project collaborators: •  Alison Kay, Ross Galloway, Heather McQueen,

Cathy Shields, Peter Kirsop (University of Edinburgh) •  Simon Bates (UBC), Morag Casey (University of

Glasgow), Kyle Galloway (University of Nottingham)

We are grateful for funding from: •  Jisc, Assessment & Feedback Programme •  HEA, Doctoral Programme

16