PeerWise: Student-generated multiple-choice questions for
enhanced engagement and learning
Judy Hardy [email protected]
Institute of Physics Higher Education Group Technical Meeting, Edinburgh, October 2013
2
PeerWise: Motivation
3
Time on task
For a course with N contact hours per week, how much time do you spend on self-study? 1. ~0 2. N/2 3. ~N 4. 2N 5. >2N
How much time do think you should spend on self-study? 1. ~0 2. N/2 3. ~N 4. 2N 5. >2N
4
PeerWise
• Web-based MCQ repository – Free to use, developed by Paul Denny, University of Auckland
• 2007 – 2010: > 40 institutions > 20,000 students > 50,000 questions > 1,000,000 answers
• 2007 – 2013: > 700 institutions > 100,000 students > 500,000 questions > 10,000,000 answers
5
PeerWise
• Content built by students: – Develop questions with associated explanations – Answer questions written by their peers – Rate questions for quality and difficulty – Take part in discussions with question author – Follow other authors – All activity is anonymous
6
Example Implementation
• Physics 1A: PeerWise replaces two weekly handins • Activity introduced in workshop session
– Students worked through structured example task and devised own questions in groups
• Two assessment deadlines – But available to the end of the course
• Counts 4% of total course mark • Minimum requirements
– Write 2 questions – Answer 10 questions – Rate & comment on 6 questions
7
Implementation cont.
• We are deliberately hands off: – No moderation – No corrections – No interventions
• But we observe…..
8
Student contributions
• 5 courses from 3 science disciplines at 3 universities – Physics, Chemistry, Biology / Edinburgh, Glasgow, Nottingham – ~850 students and ~10 members of staff
9
Student contributions
• Participation above minimum requirements • Average quality generally good
– Problems not ‘bookwork exercises’ – Few trivial questions / nonsense distracters
• Comments on questions constructive – ‘Community moderation’ corrected sub-par questions
• (Usually) didn’t leave everything to the last minute
10
End of course performance (Physics 1A)
11
PeerWise activity Effect sizea
Number of questions authored 0.24***
Number of questions answered 0.24***
Average comment length (chars) 0.22**
Days of activity 0.33***
aSpearman’s partial correlation coefficient between PeerWise activity and end-of-course exam mark, controlling for pre-course FCI mark **p<0.01 ***p<0.001
End of course performance
12
Learning networks
13
Summary
• High level of engagement – Both number and quality of contributions – Scaffolding is important
• Positive correlation with end-of-course performance – Even after taking prior ability into account – Complex pattern in terms of types of activity – But don’t underestimate time on task
• Not specific to discipline or institution
14
For more information
PeerWise http://peerwise.cs.auckland.ac.nz
15
PeerWise Community http://www.peerwise-community.org
Acknowledgements
Thanks to the project collaborators: • Alison Kay, Ross Galloway, Heather McQueen,
Cathy Shields, Peter Kirsop (University of Edinburgh) • Simon Bates (UBC), Morag Casey (University of
Glasgow), Kyle Galloway (University of Nottingham)
We are grateful for funding from: • Jisc, Assessment & Feedback Programme • HEA, Doctoral Programme
16