using video lecture capture (vlc) systems to enhance teaching in high dfw courses at wssu: case...

WSSU – Winston‐Salem State University

CETL – Center for Excellence in Teaching and Learning

VLC

Using Video Lecture Capture (VLC) systems to enhance teaching in high DFW courses at WSSU: Case studies

Dr. Alvaro H Galvis, [email protected], CETL Director and VLC project leader Dr. David Kump, [email protected], Life Sciences Department Dr. Jeffrey Overholt, [email protected], Life Sciences Department Dr. Nelson Adams, [email protected], Behavioral Sciences and Social Work Department

Winston‐Salem, August 2010

2

Using Video Lecture Capture (VLC) systems to enhance teaching in high DFW courses at WSSU: Case studies

Table of Contents Participants ................................................................................................................................................... 3

Study leaders ............................................................................................................................................. 3

Collaborators ............................................................................................................................................. 3

Abstract ......................................................................................................................................................... 4

About VLC and Panopto ................................................................................................................................ 5

Literature review ........................................................................................................................................... 6

Pilot studies in two phases ........................................................................................................................... 8

Educational findings from the first phase of the pilot study .................................................................... 9

Educational findings from the second phase of the pilot study ............................................................... 9

In depth studies about the use of PANOPTO in high DFW classes ............................................................. 10

Understanding the three VLC interventions ........................................................................................... 10

Data collection and processing ............................................................................................................... 12

Findings from the use of VLC in the three courses ................................................................................. 12

Final survey results .............................................................................................................................. 12

Correlation between student view of VLC and scores in related tests ................................................... 16

From PSY 2301 data in Panopto log system, quizzes, and exams ...................................................... 17

From BIO3311 data in Panopto log system and exams ...................................................................... 17

From BIO 1301 data in Panopto log system, quizzes, and exams ....................................................... 19

Lessons learned from VLC experience .................................................................................................... 21

Conclusions and recommendations ............................................................................................................ 22

Works Cited ................................................................................................................................................. 24

3

Participants This study was conducted by the Center for Excellence in Teaching and Learning (CETL) from Winston‐

Salem State University in collaboration with faculty members from the College of Arts and Science and in

coordination with the Information Resources Office.

Study leaders Dr. Alvaro H Galvis, [email protected], CETL director and VLC project leader Dr. David Kump, [email protected], Life Sciences Department Dr. Jeffrey Overholt, [email protected], Life Sciences Department Dr. Nelson Adams, [email protected], Behavioral Sciences and Social Work Department

Collaborators Dr. Ian Toppin, [email protected], faculty developer CETL and VLC technology leader Dr. Naomi Hall, [email protected], Behavioral Sciences and Social Work Department, and VLC evaluation leader Ms. Jigishaben Patel, [email protected], CETL graduate assistant, data and video processing

4

Abstract This in‐depth study is part of a systematic effort to improve teaching effectiveness in courses that

traditionally show high D, F, W grades (high DFW) at Winston‐Salem State University (WSSU). A pilot

study in two phases was designed and conducted during summer 2009 and fall 2009 semesters in order

to solve important questions concerning the educational use of Panopto. Panopto is a Video Lecture

Capture (VLC) system that was selected among others as a good candidate because of its functionality,

robustness, and affordable operational costs. The pilot study’s first phase (summer 2009) confirmed

that Panopto was a robust and functionally effective VLC system, both for teachers (capturing and

sharing video lectures) and students (having access and supplementing their course study materials with

VLC learning objects). The second phase (fall 2009) concerned questions about students’ and teachers’

opinions regarding VLC in different courses. The second phase also assessed VLC’s impact on student

performance and whether students viewed or did not view VLC lessons on a regular basis, as well as

statistical data analysis to corroborate findings from similar studies in other organizations. Three

courses with different methods of using VLC were compared with the purpose of learning from each of

the experiences and determining VLC conditions that might generate positive results. Data from the

second phase was also analyzed in this in‐depth study. Participating faculty were interviewed by the

study leader and their reflections about processes and products helped to understand data from student

surveys, from Panopto’s log system, and from each faculty’s grade system. Lessons learned by students

and faculty serve as framework for conclusions and recommendations for teachers willing to use VLC

systems such as Panopto. It was established that the use of VLC to enhance teaching can be very

productive when students review video recordings on a regular basis; finding the way to intrinsically

motivate students to benefit from video recordings seems to be the great challenge that faculty might

have using VLC systems. Students who regularly viewed videos to supplement other media available in

their courses got better exam results than students who did not view videos, although this fact did not

always show with significant differences to non‐viewers. However, impact on quizzes’ results was not

evident. On the other hand, the amount of time reviewing video lectures seemed to matter, as it

appeared to correlate with exam scores in one of the courses. What, when, and how to view the

recordings also seemed to make a difference, as suggested by student feedback. Generalized student

opinions about the convenience of video lecture capture as a supplement to other media used in their

courses can be attributed to important reasons mentioned by respondents concerning why it was worth

viewing recorded lectures. Also, high opinion of students about VLC may be explained by the way VLC

was integrated in the course, as it is evident in BIO 3311, where active learning before, during, and after

class activities served to improve overall student performance. Based on this experience and on

literature review findings, authors provide suggestions for possible future users of VLC.

5

About VLC and Panopto Video Lecture Capture (VLC) technology allows faculty to record their lessons and share them on the

Internet with their students, including audio (such as the faculty member’s lecture), computer feeds (the

presentation or screen capture), and video if desired (the faculty member explaining or doing work at

the smart board/whiteboard). Lessons can be recorded alive during class session or out‐of‐class (before

or after class time) and can be uploaded to a web server with stream‐video capacity. The student can

then get access to the recording via a Learning Management System such as Blackboard and review each

class session at his/her own pace, searching contents via text, video thumbnails, time selection, or

sequential order. The VLC system keeps a record of who views what, when, and for how long.

After reviewing commercial and open source VLC systems, CETL and the Office of Information Resources

at WSSU selected Panopto (see http://www.panopto.com ) to pilot test VLC systems in a face‐to‐face

setting. Panopto is an open source VLC system that can be used at a reasonable price. It allows recording

of class sessions from networked computers with webcam and microphone, screen casting with audio

and video almost immediately after the recording, hosting the information locally or remotely, and

sharing the video efficiently on the Internet at costs that are significantly lower than commercial VLC.

Experience at Suffolk University implementing Panopto’s Course cast system indicates that this solution

allowed people to record anytime, anywhere with standardized technology. In higher education, “there

is very little IT involvement during the recording & publishing process. We are giving the end‐user the

power to manage it all. Relevancy of content depends on how quickly it can be made available.

CourseCast provides that ability” (Panopto, 2008). Based on the above considerations and having

conducted hands‐on exploration of the product, WSSU received a free 45‐day trial license of Panopto

during the summer of 2009.

Three faculty members agreed to use Panopto to supplement instruction during their summer 2009

courses; these faculty members recorded and shared class sessions with their students, provided

feedback about the system and its use, and helped collect feedback from students. The summer 2009

experience confirmed that Panopto was easy to use from both the faculty and student perspectives, that

the system properly captured both screen and audio and video if requested, that it was possible to share

captured lessons via Blackboard course shells, and that the data collected by Panopto about students’

usage was useful to find out who viewed what, when, and for how long. Based on this product

assessment, a campus‐wide Panopto license was acquired at the end of summer 2009. This license was

hosted by Panopto and managed by CETL, and a VLC Initiative research protocol was submitted for

approval to the WSSU Institutional Review Board for Protection of Human Subjects. This proposal was

approved (IRB #2986‐10‐0011).

6

Literature review

A literature review about the educational impact of VLC was done as a framework for the design of the

VLC initiative, taking into consideration that the ultimate goal to integrate any technology for teaching is

to improve student learning (Galvis A. H., 2009). At WSSU, this goal is particularly important since,

according to the WSSU Strategic Plan 2010‐2015, institutional research studies have shown a constant

decline in retention each semester as a result of increasing number of students who are dropping,

failing, or withdrawing from classes each semester. As mentioned in the research protocol entitled

“Video Lecture Capture (VLC) Initiative” (Galvis, Hall, & Toppin, 2009) a number of different approaches

have been taken across the academy to address the issue of high DFWs. Some of these approaches

included modifying tutoring services, increasing mentorship initiatives, following up at‐risk students at

shorter intervals in order to monitor their progress, and providing remediation among other efforts

(WSSU, 2010). Since the core of student academic success centers around what happens in the

classroom, the lecture‐capture approach seemed to be a good solution for enhancing student

performance. Lectures captured using digital video and made available using streaming video allow

students to review information the instructor explained at their own pace. Students can also view any

associated visual aids and search for concepts and resources related to those aids. Lectures captured in

this way may help students overcome weaknesses in areas such as taking notes, paying attention over

extended periods of time, or understanding language barriers. Another very important aspect is the

students’ ability to independently review lecture material without slowing the class pace. Students who

may lack certain skills can, therefore, remediate without having to spend class time doing so.

Early work with lecture‐capture at Notre Dame University reports a positive learning impact in the ability

to review class lecture videos (Chandra, 2007). The same study also established that VLC can be used

individually by faculty members (without videographer) with sufficient quality and at reasonable cost.

Faculty concerns reported in this study (ibid) included:

(1) Students who would just listen to the videos and not attend to class itself. To our surprise, this

never happened, usually students showed up for each lecture. Student feedback collected

indicates that ‘honestly, going to lecture and seeing everything in person is much less of a hassle

than trying to catch things from a video’. Students felt that storing the videos along with the

printed notes would be more useful to give some context to the hard copy of notes.

(2) There is a record of every word spoken by faculty. Sometimes faculty may misspeak or state

something incorrectly; we consider this to be an acceptable risk, faculty are not infallible.

(3) [VLC] will take up too much of the faculty’s precious time without any tangible benefits to the

students. We made a conscious effort to not advertise the videos, the objects just showed up on

the class web page and on iTunes. All the usage reported was from voluntary users; some of the

lectures were very popular.

(4) Legality and intellectual property implications of such recordings. This is a topic for future

counsel.

7

A report by Educause Learning Initiative (2008) indicates that, while not intended as a replacement for

in‐class instruction, lecture capture systems offer three important benefits: an alternative when

students miss class; an opportunity for content review, particularly when abstruse topics are introduced

or detailed procedures are performed; and content for online course development. The same report

states that lecture capture works especially well in subject areas where students benefit from repeated

viewing of content, as when complex information is discussed or formulas are written on a board. Some

worry that students may cut classes in favor of viewing captured lectures. Yet, from the advent of the

cassette tape through the emergence of the podcast, students have found that recordings take as much

time to absorb as a live lecture, but without the opportunities for question‐and‐answer or interaction

with their classmates.

A more recent study (Schaffhauser, 2009) reports that faculty’s use of lecture capture has grown as the

systems have become easier to use. Typically, beginning the capture process requires nothing more

than “having the faculty member press a button or turn on a computer. It’s done automatically” (ibid).

Ease of use provides a threshold for turning lecture capture from an option into a necessity: “People are

going to start getting the sense that this is really helping the learner” (ibid).

Reflections about lecture capturing at University of Alberta, CA (Boufford, 2009) stated that faculty

should:

1) take advantage of using video to provide value added content to the podcast by recording the

text, images, and other visual information not just the talking head!,

2) reorganize lectures, converting them from long linear streams of content into discrete chunks of

information, where each lecture can “stand alone” on its own as a presentation,

3) put themselves in the place of a student and ask “Would I want to listen to or watch this?”

When planning to podcast lectures, always remember “presentation capture” not “talking head

capture”.

8

Pilot studies in two phases A pilot study in two phases was designed and conducted at WSSU. Phase 1 was done during summer

2009 with data provided by three faculty members who taught courses with high DFW grades and by

some of their students who used Panopto to view recorded class sessions and signed informed consent

letters. This phase allowed collecting feedback about the use of Panopto’s CourseCast system already

mentioned, as well as provided solutions to initial questions concerning students’ and faculty members’

opinion about the use of the system.

Phase 2 was conducted during the Fall 2009 semester, implementing the research protocol entitled

“Video Lecture‐Capture (VLC) Initiative” (Galvis, Hall, & Toppin, 2009). Phase 2 included eight faculty

members voluntarily selected among those who taught courses with high DFW grades. Students from at

least one of their course sections were also involved; both faculty and students voluntarily accepted to

participate in the study and signed an informed consent form. Researchers used a mixed‐method,

correlation design to investigate the following questions:

Whether this type of system can help enhance the teaching and learning processes;

If the system can enhance the teaching and learning process and to what extent;

If the availability of the videos has any significant impact on student assessment scores; and

If students find the VLC software helpful and/or valuable to their learning experience.

Once the lectures were captured by the faculty, they were uploaded to the Panopto site on the Internet.

Students and faculty in those participating classes were provided with links to Survey Monkey © and

instructions about how to access the videos and assessment tools. These included:

An initial survey to gather data about students’ background and prior knowledge (completed at

the beginning of the data collection period)

An ongoing survey to collect immediate feedback after video review

A final survey to gather feedback about perception of how using the software affected student

learning and performance in class (at the end of the data collection period)

A Faculty Assessment Form to ascertain the perceptions of using the software, faculty

assessment of the impact of student performance, and suggestions for future use.

In addition, faculty were required to provide student performance data and class assessment results

with video episodes. This data was then correlated with data captured in Panopto’s log that provided

information about who viewed which episodes and when and for how long they were viewed.

Four of the eight Phase 2 faculty were invited to participate in an in‐depth study comparing two types of

VLC‐based interventions. The first of these interventions used in‐class recording and the second

intervention consisted of out‐of‐class recording. Three faculty completed their case documentation.

Reports prepared by faculty included the information required in the VLC research protocol. In addition,

two interviews per faculty were conducted by the project PI, the first one to understand each VLC

intervention, and the second one to comment on data and findings; both interviews were video‐

recorded.

9

Educational findings from the first phase of the pilot study Findings from summer 2009 survey analysis (Hall, 2009) show that:

Students who utilized Panopto overwhelmingly (85%) thought it was helpful, beneficial and

valuable to the class in which they were enrolled.

The students who reviewed the videos more frequently, and for longer periods of time, appear

to be students whose GPA hovered around the 3.1‐3.5 range.

Two of the main factors that helped students decide whether or not to access the videos were

class requirements and time. Of those students who did not review any videos, over half said

they did not because it was not required for the course. Other students worked at least 20 hours

per week and they had to prioritize what they could do or not in this shortened summer course.

Faculty did not see the benefit of video recording full sessions for students in terms of helping

learners in their preparation for quizzes and exams. All recordings during summer 2009 were

live and class long; there was neither a requirement nor an incentive to view the recordings, just

curiosity and willingness to learn on the part of students. Recordings gave students the

opportunity to review class information with full lecture access.

It was evident from the above findings that student motivation to review recorded materials was the

most important challenge faced with the use of VLC systems. A second challenge was to motivate faculty

to find and use creative ways to include VLC in their teaching and, as a consequence, to adjust their

course syllabi to reflect activities that could promote video usage.

Educational findings from the second phase of the pilot study Findings from fall 2009 survey and performance assessment data (Hall, 2010) indicate that:

The vast majority of students thought the VLC was helpful, beneficial, and valuable to their learning experience in the participating classes.

The students who accessed the VLC system most frequently were those who had GPAs either in the very high or low range.

Students who worked more than 11 hours per week were more likely to utilize the videos for instruction and review.

The availability of pre/post‐recorded and live recorded videos did not negatively impact student attendance in classes.

Data indicate that class structure should be taken into account when deciding whether or not to implement VLC in a particular course.

VLC usage did not have a statistically significant impact on assessment scores in individual classes; however, the impact of usage on assessment scores was seen when data was aggregated.

10

In depth studies about the use of PANOPTO in high DFW classes Three faculty members participated in the in‐depth study. The study was designed to establish the

impact on students and faculty who used two types of Panopto‐based VLC interventions: live class

sessions (recorded during class sessions), out‐of‐class lecture sessions (pre or post‐class recorded

videos).

Understanding the three VLC interventions The following table helps to understand VLC intervention in each case. In two of the cases, BIO 1301 and

PSY 2301, VLC was the only digital resource added to the “normal version” of the course, including both

class activities and tests; in BIO 3311, VLC was one of several digital components embedded in the

course activities and assessment system. In the three cases faculty made use of genuine in‐class

dialogues around topics explored and discussed in class. VLC was used live and out‐of‐class, with

different roles and goals as presented in the following table.

Course code and name

Course characteristics and instructor Methods, number and length of recordings

BIO 1301 General Biology

BIO 1301 is a science course required for all WSSU students. The course consisted of two weekly lecture periods and a weekly lab period. Freshmen students were predominantly course participants; class size was nearly 90 people for lectures and around 30 for labs. Dr. David Kump is an assistant professor of physiology at the Life Science Department. This was the first time he taught BIO 1301 and he was developing his particular approach to teaching this class. Over the 4th part of the semester, he introduced interactive chalk talks as a means to foster genuine dialogues about concepts visualized either on the white board or on the Power Point.

Eleven classes were recorded live and belong to nearly 75% of the course content; they are related with exams 1 to 3. Average video length was 35 minutes. Power points provided by text publisher served as lecture organizers. Two preparation quizzes (2 minute and 16 minute videos) were recorded out‐of‐class, the first one similar to weekly text‐based quiz preparation messages, the second more lecture‐type but focused on key elements. Four short videos, related to nearly 25% of the course content, and associated with exam 4, were also recorded alive. Average video length is 17 minutes. Interactive “chalk‐talk” with diagrams and notes on the white board, plus power point presentation of textbook‐based figures, served as organizers of video lectures captured.

PSY 2301 General Psychology

PSY 2301 is designed to provide the student with a broad knowledge of the science of human behavior and consciousness. It is primarily a lecture course where questions and discussion are encouraged. Two sections of PSY 2301 participated in this research. Of

Dr. Adams decided to record supplemental video segments as enrichment type presentations dealing with topics potentially difficult to students. Based on his classroom interaction, Dr. Adams choose topics that added value to class sessions and provided further explanations about selected slides

11

some 80 students, nearly 65 made use of video recordings to supplement their study, but only 36 students gave permission to use their data. Dr. Nelson Adams is a professor of Psychology at WSSU. He has taught many different Psychology courses in the Psychology curriculum. He was awarded the UNC Board of Governors Excellence in Teaching Award in 2002.

used in class. Through the Fall 2009 semester Dr. Adams recorded twelve videos in his office. Each recording was made available on the PSY 2301 course site in Blackboard. Students were then able to look at a short presentation of what had been discussed in class.

BIO 3311 Fundamentals of Anatomy and Physiology, and accompanying Lab (BIO 3111)

BIO 3311 course and accompanying Lab BIO 3111 were redesigned for active learning and technology integration during Summer 2009 and were taught during Fall 2009. BIO 3311 is a junior and senior level course for biology majors that typically enrolls 40‐50 students. The course had been previously taught using traditional lecture approach with standard paper quizzes and assignments, with a 40% to 50% failure rate. In the Fall semester 2009, the semester of the redesign, 38 out of 40 students enrolled completed the course. Dr. Jeffrey Overholt is Assistant Professor of Physiology and a member of the Biomedical Research Infrastructure Center of the Life Sciences Department at WSSU.

BIO 3311 redesign included several components, one of which is VLC, as follows:

Learning objectives per course unit

YouTube video materials to provide a good context per course unit

Lecture Power Point materials available for review

Pre‐class online quiz to be graded

VLC available after class or lab ended

Practice quizzes available online

Flashcards / crossword puzzles / games from publisher’s material.

VLC live recordings promoted active learning outside of the classroom. Recording live sessions allowed students to review questions raised in class, as well as to reflect on discussions about concepts addressed through class quizzes. BIO 3111 Lab sessions required students to prepare answers to any of 30 questions per lab posted in Blackboard (Bb). Randomly selected students presented their answers before the class using VLC. Immediate feedback was obtained using interactive devices (clickers).

12

Data collection and processing Students participating in above‐mentioned courses were invited to sign informed consent forms which

they returned to their teachers at the beginning of the semester; in addition, respondents to the final

survey were asked to confirm their permission to use data in VLC studies and publications. Survey

instruments designed for the VLC initiative (see Phase 2 pilot study above) were used to collect

information from students and faculty concerning their opinions about the use of VLC to supplement

course materials. On the other hand, Panopto’s log system provided information about who viewed

which video lecture and when and for how long the lecture was viewed. In addition, each faculty

member extracted information about performance in quizzes and exams from the course grading

system. With that information, each faculty member analyzed the data and produced a report that

supported the following findings and the creation of three video papers, in collaboration with Dr. Galvis

from CETL (Adams & Galvis, 2010), (Overholt & Galvis, 2010), and (Kump & Galvis, 2010).

Findings from the use of VLC in the three courses Three sets of data helped evaluate the use of VLC by students and the impact it could have on student

learning: (1) answers to final survey, (2) log data from Panopto system, and (3) assessment data from

course grading system.

Final survey results A subset of students who ended each of the courses accepted the invitation from their teachers to give

feedback about the experience. Participation points were offered to those who answered the survey;

students were allowed to authorize or not use of their answers in this study. Major findings from

authorized answers to this survey include:

Use of VLC video varies, depending on the type (in class, out‐of‐class) and length of the

recording (class period, segments). There are more “heavy viewers” (watched more than 5 VLC)

in BIO 3311 (live full class recordings) than in PSY 2301 (short post recordings) or BIO 1301

(combined recording methods).

The great majority of students (nearly 80% of respondents) expressed a high positive impact on

conceptual understanding because of VLC video viewing. BIO 3311 respondents rated the

impact of viewing videos higher than respondents from the other two courses.

The great majority of students (nearly 80% of respondents) declared that there was no impact

of viewing VLC videos on class attendance. In BIO 3311 answers to this question were

unanimous.

There are important suggestions from students concerning why, what, when, and how to view

VLC videos. These suggestions take into consideration the type of videos made available.

From final survey: Use of VLC by students The final survey asked, “How many different video lectures did you view during this course?” The

following chart helps visualizing answers, as well as the differences among them. While PSY 2301 and

BIO 1301 respondents fall in all levels of VLC use, with nearly 20% of heavy viewers (more than 5 video

lectures), all BIO 3311 respondents viewed at least one VLC, with more than 50% of heavy viewers. The

predominant level of use seemed to be dependent on the course, but there was not a trend.

13

Figure 1. Number of video‐lectures per course watched during fall 2009 semester

In the three courses instructors asked students to review videos to supplement class notes and text

information; credit for watching the videos was not given in any of the cases. In all cases, students had

video lectures available in Bb almost the same day the recording was done, and videos were available

24/7 after they were posted in the corresponding Bb course shell.

From final survey: Time spent viewing video lectures The final survey asked, “On average, how much time would you say you spent reviewing each

video?”The following chart helps visualize end‐of‐course (EoC) answers. While BIO 3311 videos

captured the whole class session and nearly 60% of respondents viewed 20 or more minutes or the

entire video, in PSY 2301 videos were 10 to 20 minutes long and nearly 50% of respondents viewed 20

or more minutes or the entire video. In BIO 1301, 75% of video lectures were class long while 25% of the

video lectures were segments of class sessions; nearly 35% of respondents in BIO 1301 viewed 20 or

more minutes or the entire video.

Figure 2. Time spent viewing video lectures per course

0% 20% 40% 60% 80% 100%

BIO 1301 (EoC respondents = 32 …

BIO 3311 (EoC respondents = 29 …

PSY 2301 (EoC respondents = 39 …

0 1‐2 3‐4 More than 5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

BIO 1301 (EoC respondents = 32 out of 90)


PSY 2301 (EoC respondents = 39 out of 80)

Less than 5 minutes eachBetween 6‐10 minutes eachBetween 11‐20 minutes eachBetween 21‐30 minutes eachMore than 31 minutes each/entire video

14

From final survey: Students’ opinion of VLC The final survey asked, “On a scale from 0‐10 (10 being the highest), how beneficial were the videos

viewed to your understanding of the concepts during this course?” The following chart helps visualize

EoC answers, as well as differences among them.

Figure 3. Students’ opinions of VLC

In each case there was a predominant positive impact on conceptual understanding because of VLC

video viewing (nearly 80% of respondents valued contributions with 6 or more points out of 10 in all

courses). Participants in BIO 3311 valued the contribution of VLC viewing higher than participants in the

other two courses. The weighted average in BIO 3311 was nearly one point higher than the W‐AVG for

the other two courses.

From final survey: Impact of viewing VLC videos on class attendance The final survey asked, “The availability of the videos had…. [effect on attendance].”The following chart

helps visualize EoC answers, as well as differences among them. In all cases more than 80% of

respondents declared that there was no impact of viewing VLC videos on class attendance. In BIO 3311

that response was almost unanimous.

Figure 4: Impact of viewing VLC on class attendance

0% 20% 40% 60% 80% 100%

BIO 1301 (W‐AVG = 7.09 points, 32 answers)

BIO 3311 (W‐AVG 8.14 points, 29 answers)

PSY 2301 (W‐AVG 7.26 points, 39 answers)

0 to 2 pts

3 to 5 pts

6 to 8 pts

9 or 10 pts

0% 20% 40% 60% 80% 100%



PSY 2301 (EoC respondents = 39 out of 80)

No effect on my attendance

Increased my attendance

Reduced my attendance

15

From final survey: Suggestions to benefit from video usage The final survey invited students to “provide any suggestions to other students on how to use the videos

to their benefit.” The following excerpts from student responses illustrate why, what, when, and how to

view VLC videos.

Table 1. Why view VLC videos?

PSY 2301 BIO 3311 BIO 1301 He breaks down everything better than he does in class. It's long but it's helpful

It is very beneficial if you missed class or if you didn't understand a certain section or missed something in lecture. It could possibly answer any question that a student might have.

Teachers will say things in them that you may have missed that will be on a test or quiz and you would never know it.

The professor goes more into detail, uses more examples than he may during class, so it helps you understand concepts better than you may by just attending class

View them regularly and after each class in order to make sure you have a clear understanding of what was discussed in class

It’s easy to get materials well and you can always go back to listen to an area you did not understand rather than in class where once said its gone, unless you ask a question for clarity

Table 2. What to view in VLC videos?

PSY 2301 BIO 3311 BIO 1301 Watch the entire video to understand all the content of what is being viewed

The videos come in very handy when you are reviewing for an exam. You can go back after re‐reading the chapter and pick up on the concepts you didn't quite understand from the reading. And also if you seem to have forgotten something talked about in the lecture it helps to refresh your memory. It's a very good tool to have for class.

Watch the whole video over and over until I get full understanding of the concepts

Allows an easy review of current and past information on particular topics.

The videos can be used to further reiterate concepts that were taught in the lecture.

Table 3. When to view VLC videos?

PSY 2301 BIO 3311 BIO 1301 Watch them before every quiz and test. Also take notes when watching them.

You should always review the course material before class. After class watch the videos to comprehend something that you may have had trouble with in class. Also before exams you can use the videos to review the course material.

Don’t use them as a substitute for coming to class! The videos are an excellent tool to refresh yourself, but you lose the opportunity to ask questions or expand on topics if you aren’t in class.

16

If you are absent from a class and you need to know what was discussed the videos are very helpful.

I believe the video lectures help the most when students are absent. If the day's lecture was missed one could get on the website and watch and listen to that specific lecture.

I think that students will benefit more from the video if they expect to learn something or have a idea clarified, if you go in blind I believe you come out that way

I turn on the video lectures and listen to them while cleaning and doing other things.

After every lecture, watch the video. It is really beneficial

Table 4. How to view VLC videos?

PSY 2301 BIO 3311 BIO 1301 When watching the videos be sure to take notes and pay attention as if you were in class to get an understanding of what is being said.

It is a good idea to review your notes then watch the video then review your notes again to get a good understanding of the material.

View the videos and rewind to material that is significant to passing tests/quizzes

Take notes while watching the video. I comprehended the material better after attending the class and watching the videos before tests.

The videos would not be helpful to me unless I read the text and followed along the text with the power points and then review the video for more understanding if I still didn't clearly grasp the information.

Correlation between student view of VLC and scores in related tests CETL provided each faculty participating in the in‐depth study data from Panopto’s log system

concerning who viewed what recording and for how long. Faculty used this information and scores in

their grading system to find out if there were differences among students who effectively were

“viewers” and the rest of the class. These are the findings:

In all three courses, students viewing videos systematically received better exam grades than

non‐viewers. In PSY 2301, performance improvements in non viewers and viewers comparing

final exam means to the previous exams means, were significantly different and I were in favor

of viewers. In BIO 3311, time spent by regular viewers of VLC lectures appeared to correlate well

with exam scores. In BIO 1301, differences between non‐viewers and viewers were not

significant. In all cases these positive results must be treated with caution since the number of

students who regularly viewed videos was relatively small, and thus further work is needed to

replicate this effect.

Quiz comparison between viewers and non‐viewers did not show significant differences in PSY

2301 and BIO 1301.

17

From PSY 2301 data in Panopto log system, exams and quizzes Dr. Adams assessed the effectiveness of VLC lecture by looking at differences among viewers and non‐

viewers who had signed informed consent forms. The overall results showed that among a small subset

of students who viewed videos, there was a clear sign of better performance on the final exam and on

the quizzes than among the non‐viewing students.

Analysis of exams in PSY 2301 Dr. Adams compared 38 students who did not view videos during the second half of the course versus 7

students who viewed some or all of relevant videos for the final exam. He found a significant difference

of means between non viewers (mean 70 pts, standard deviation of 14.8, standard error of 2.43) and

those students viewing videos (mean 83.9 pts, standard deviation of 6.0, standard error of 2.45); t test,

t(43) = 4.02, p < 0.001. He also contrasted improvements in non‐viewers and viewers comparing final

exam means to the mean of Exams 1 and 2 and found that improvements in non‐viewers was 1.94 with

standard deviation of 10.8 (standard error of 1.8) and that the improvement in viewers was 12.5 with a

standard deviation of 7.5, standard error 3,1. This comparison was significantly different, t(43) = 2.97, p

< 0.01.

Analysis of quizzes 58 in Psy2301 When each quiz was compared between viewers and non‐viewers there were no significant differences

for any quiz, although in each case the viewers had higher scores on an absolute basis.

Discussion of results in PSY2301 Dr. Adams mentioned that the above positive results must be treated with caution for at least two

reasons. First, the number of students who viewed videos was relatively small, and thus further work is

needed to replicate this effect. Second, the potential effect of general motivational factors cannot

separated out; it is possible that viewing students were more motivated in general and thus performed

better regardless of video watching.

From BIO3311 data in Panopto log system and exams Dr. Overholt analyzed data from 39 individual students who watched videos in the Panopto system for a

total of 7719 minutes, and found that use of the system was dominated by 7 students who, combined,

watched 5528 minutes. This observation was used to establish a possible relationship between Panopto

use and exam scores.

Analysis of exams in BIO 3311 The following figure shows the average exam scores from the 7 regular VLC users compared to that of

the remaining students. First, it can be seen that the average exam scores for all 4 exams averaged over

12 points higher for the regular VLC users than the rest of the class. Second, it appears that as VLC use

increased for the regular users, exam scores increased. In contrast, the average exam scores fell off at

the end of the semester in the remainder of the students

18

Exam 1 Exam 2 Exam 3 Exam 4

Ave

rage

Exa

m S

core

s (m

ean+

/-S

E)

55

60

65

70

75

80

85

90Non-users (n=32)Regular Panopto Users (n=7)

Exam 1 Exam 2 Exam 3 Exam 4

Min

utes

/Stu

dent

0

50

100

150

200

250

300

350Non-users (n=32)Panopto Users (n=7)

Figure 6. Average exam scores in Bio 3311, comparison viewers / Non viewers

The following figure plots the number of Panopto minutes viewed before each exam by each of the 7

individual regular Panopto users and their exam scores. The overall correlation between increasing use

of Panopto and exam scores can be seen as the red regression line derived from the plots of the 7

students. Interestingly, the minutes viewed by the regular users appeared to correlate very well with the

exam scores.

Panopto Minutes

0 100 200 300 400 500 600

Exa

m S

core

(%

)

60

70

80

90

Student 1Student 2Student 3Student 4Student 5Student 6Student 7Regression

Figure 7. Number of Panopto minutes viewed by “regular viewing” VLC videos students

Discussion of results in BIO 3311 While figure 6 shows that the 7 regular Panopto users had an average exam score over 12 points higher

than the rest of the class, this can be interpreted in at least 2 different ways. First, many times it is the

best students who take advantage of all available resources and this observation could skew the

statistics in favor of the Panopto users. This may be further supported by the fact that, even on exam 1,

when there was not much Panopto use, exam scores for the regular Panopto users was nearly 13 points

19

higher. However, looking at all of the data shows that, even for the regular Panopto users, exam scores

increased in parallel with Panopto use (see inset) and remained higher, while scores in the rest of the

class trended downward.

In conclusion, figures 6 and 7 suggest that VLC use improved student exam scores. Whether or not this

data is convincing, the most important things are: 1) student comments were unanimous in that they

really liked the new material and technology in the redesign; and 2) students used the material,

especially Panopto (7719 minutes). Perhaps it was expressed best by the student who commented: “I

have enjoyed his class a lot. The interactive learning was essential for my success in the class.”

From BIO 1301 data in Panopto log system, quizzes, and exams Data captured in the Panopto log from seventeen videos recorded from Dr. Kump’s lectures were used

to analyze the impact of viewing videos on student related assessments. Results in exams 1 to 3 were

studied in regard with recording method 1 (live class sessions), results in exam 4 in regard with

recording method 3 (live segments of class sessions) and results in quizzes 1 and 7 in regard with

recording method 2 (pre‐recorded preparation for the quizzes).

Video capture usage Of the eleven videos captured from lectures with method 1, each was accessed by an average of 9±2

individual students, with an average access time per student of 7±2 minutes each. Based on the number

of students enrolled in the course at the time each video was captured, an average of 10±0% of enrolled

students accessed each video. For method 2, two videos were designed to help students prepare for

quizzes. The first video was a two‐minute video that prepared students for quiz 1 and served to

introduce them to the Panopto system; 73% of the students watched the entire 2‐minute video and 13%

of the students watched only a portion of the video. The second video was a 16‐minute video that

prepared students for quiz number 7; 32% of students watched almost all the video, 24% watched part

of the video, and 44% did not watch the video at all. Of the four videos prepared with method 3, ranging

between 8 and 27 minutes each at an average of 17±5 minutes, an average of 11±1% of students

accessed the videos. The above low level of video usage was not surprising, as a review of Blackboard

reveals that only about half of the students accessed the course shell during any one‐week period.

Effect of video capture usage on exams and quizzes Students who viewed VLC performed better in all exams, as shown in figure 8 below. However, a two‐

way Anova indicated that video watching accounted for less than 0.1% of variance in exam scores, with

the exams responsible for approximately 5% of the variance; there was no correlation between

watching videos and exam scores (p=0.59).

20

Figure 8. Mean scores of students not watching video (white bars) and student watching video (black bars) for each exam in BIO 1301.

Comparison of exam results between method 1 and method 3 show that the mean score for exams 1‐3

(66.5 ±1.0) was significantly lower than the mean for exam 4 (76.7 ±2.0). Considering that video usage

both with methods 1 and 3 was similarly low, this supports the idea of recording shorter and focalized

live videos rather than full class sessions.

A comparison of quiz scores between quizzes that had a preliminary video (quizzes 1 and 7, mean of

73.7 ±4.3) and those for which students were directed to their textbooks (quizzes 2‐6, and 8; mean 72.0

±1.0) shows that there is no difference in quiz scores when videos are used (p=0.21).

Discussion of results in BIO 1301 Student achievement and success ultimately depends upon what students do outside of the classroom.

High‐achieving students are typically those who spend the necessary time and effort outside of class

studying effectively. Effective study sessions include reviewing notes and studying the textbook.

Panopto offers an additional method for students to use their study time reviewing what was discussed

during class.

Based on survey results, students agreed that the captured videos were as effective as an instructional

technique; they also agreed that reviewing the captured videos helped them to learn and understand

the material. The exam results of video watchers seemed to be higher than those of non‐video viewers,

although there was not a significant statistical difference. This consensus and results, in the context of

the relative low usage of videos on a regular basis, raise significant and important questions, not only

concerning Panopto, but regarding the motivation for students studying outside of the classroom. It may

be that students only turn to videos for further help with a specific concept, using them as a supplement

to more traditional methods.

21

Lessons learned from VLC experience Instructors participating in the in‐depth studies shared their insights concerning the processes and

products obtained. Through interviews it was established that:

1. All of the instructors found Panopto’s VLC system easy to use, independently of whether they

were recording live or out‐of‐class. Setting up the system for recording required a few minutes

of technology preparation, and video uploading and sharing was straight forward.

2. Dr. Adams found that recording lectures in front of the camera and without audience helped

him develop verbal fluency delivering multimedia messages. He recommends this experience to

colleagues who may feel uncomfortable and shy away from the idea of being videotaped.

3. Dr. Adams wonders how to expand the impact of videos on students learning: if some students

clearly benefitted from watching videos, how can the entire class be engaged in this process?

He believes that tailoring videos to student needs has shown to be an alternative to keeping

students’ initial motivation towards video viewing. He considers that making the video setting

more attractive than his office could be another path to explore.

4. Dr. Overholt found that course redesign with technology is easy and effective, but it requires

careful design, setup, and technology testing to overcome problems that might arise. He

integrated several technologies into his course, including clickers, practice digital quizzes,

games, and Panopto. These tools helped recording live and sharing online in‐class discussions

and in‐lab student presentations. Data collected suggest that Panopto usage improved users’

exam scores, and the overall course redesign improved student performance in class, engaging

them as active learners.

5. Dr. Kump considers that Panopto should be used to record short topic‐specific videos that

supplement students in their personal study after thoroughly reviewing their lecture notes and

assigned material from the text. If he were to redesign the course he would like to find ways to

motivate students to view those videos. Panopto is a tool that students can use outside of the

classroom to go back and review material; it is not a substitute for the textbook or study notes.

Panopto is a supplement; it is not the only tool available to understand concepts, but any tools

are only effective if students use them.

22

Conclusions and recommendations The use of video lecture capture to enhance teaching can be very productive when students review

video recordings on a regular basis; finding the way to intrinsically motivate students to benefit from

video recordings seems to be the great challenge that faculty might have using VLC systems. Data

collected in this study suggest that students who regularly viewed videos to supplement other media

available in their courses got better exam results than students who did not view videos, although this

fact did not always show with significant differences to non‐viewers. However, impact on quiz’ results

was not evident. On the other hand, the amount of time reviewing video lectures seemed to matter, as

it appeared to correlate with exam scores in one of the courses. What, when, and how to view the

recordings also seemed to make a difference, as suggested by student feedback. Generalized student

opinions about the convenience of video lecture capture as a supplement to other media used in their

courses can be attributed to important reasons mentioned by respondents concerning why it was worth

viewing recorded lectures. Also, high opinion of students about VLC may be explained by the way VLC

was integrated in the course, as it is evident in BIO 3311, where active learning before, during, and after

class activities served to improve overall student performance.

Based on this experience and on the literature review findings, we provide the following

recommendations to potential users of VLC systems:

1. Before making a decision concerning the use of Panopto to provide supplemental instruction in

your course, play with the system and analyze what features could be useful in your case.

Panopto has basic options (voice and screen image capture) and also advanced features (video

capture, annotate presentation). You can share full class sessions on the Internet, or segments

of class sessions (you can edit recordings) for students to view using a web browser. You can

also share voice cast (MP3 files) and/or video cast (MP4 files) for students to download and view

with appropriate digital players. Depending on course needs, set up Panopto at your

convenience.

2. When you have recognized that Panopto’s features are worth using to offer out‐of‐class

supplemental instruction to your students, try to establish what the best way to use Panopto in

each selected course may be. The following elements are worth considering:

a. You can record live (in class) or out‐of‐class (pre‐ or post‐class). The key element is to

add value to students’ experience, and this deals with capturing things that go beyond

what is already available on other media (text, power points, handouts). Live class

sessions, or segments, were found relevant as long as they captured discussions or

interaction that clarified key points, and/or allowed students to review explanations

about concepts when they were preparing for exams or quizzes. Pre‐quiz recordings

were considered useful as long as teachers said things that students may have missed

and that might be on a test or quiz. Post‐class recordings about specific topics were

found useful as long as professors went into details, helping students to understand

difficult concepts.

b. You can upload either the whole recording, or segments of it to the VLC server. First,

you let the system convert the information to screen cast format and later on the

23

corresponding URL can be published; Secondly, you should use Panopto’s editor and

select what you want students to be focused on. You can also delete segments of the

discussion that deal with privacy issues or that have dead time recordings. Remember

that Panopto allows students to search by using key words, notes, time stamps, or

thumb nails; with these features students have direct access to the segments of their

interest.

3. Be sure that your students know how to use digital resources that you generate or allocate from

VLC recordings. Take the following suggestions into consideration:

a. In spite of the fact that many students are computer literate, novices in the use of

Panopto (or any other VLC system) will need induction to the system. Demonstrate the

system in your classroom with a video projector, and/or conduct a hands‐on experience

session at a computer lab once you have a VLC session available. Be sure that students

learn how to get access to VLC lessons you have captured, how to adjust the screen

(zoom in/out), and how to search by topic, note, time stamp, or thumb nails. Taking

notes while reviewing a lesson is an important feature; be sure to demo it. If you want

students to download and use MP3 (audio files) or MP4 (video files) from their mobile

devices, include a demo introductory session of this as well.

b. Video lectures should be available in a very simple and direct way. Panopto allows

sharing the link to VLC recordings through a direct link generated with Panopto’s feature

from your LMS activity editor. MP3 and MP4 files should be downloadable directly from

the Panopto server. You can get the links posted in your learning management system

for students to download. Students can also subscribe to your Panopto folder feeds

using the corresponding RSS link.

4. Finally, but not least important, be sure to comply with privacy, copyright / intellectual property

issues. If you are using graphics, images, and/or audio/video clips available on Internet, obtain

copyright clearance if these items are copyrighted, or use the Creative Commons license

attributes when applicable. If your VLC will include student contributions (e.g., presentations,

responses or questions), you should ask students to sign a consent form when the podcast

audience is broader than the class itself; however, when the podcast is put on a secured website

for only class to view, no release is required.

24

Works Cited Adams, N., & Galvis, A. H. (2010). After‐class recorded videos to supplement instruction in General

Psychology. Winston‐Salem, NC: WSSU ‐ CETL (multimedia). Boufford, B. (2009, May 15). Don’t be a Talking Head: Presentation capture not lecture... Retrieved June

12, 2010, from Ask Dr. C's ‐ ADC Blog: http://blogs.elearning.ualberta.ca/wpmu/ecast/2009/05/15/dont‐be‐a‐talking‐head‐presentation‐capture‐not‐lecture‐capture/

Educause. (2008, December). 7 things you should know about.. Lecture Capture. Retrieved June 12, 2010, from Educause Learning Initiative: http://net.educause.edu/ir/library/pdf/ELI7044.pdf

Galvis, A. H. (2009, May). Establishing a PLACE for teaching technologies. Retrieved June 15, 2010, from Google sites: http://sites.google.com/site/placetools/

Galvis, A. H., Hall, N., & Toppin, I. (2009). Video Lecture‐Capture (VLC) Initiative. Winston‐Salem, NC: WSSU ‐ Center for Excellence in Teaching and Learning.

Hall, N. (2009). Video lecture capture initiative: summer pilot final report. Winston‐Salem, NC: Winston‐Salem State University,Center for Excellence in Teaching and Learning.

Hall, N. (2010). Video lecture capture initiative ‐ Fall 2009 initiative report . Winston‐Salem, NC: Winston‐Salem State University, Center for Excellence in Teaching and Learning.

Kump, D., & Galvis, A. H. (2010). Video‐enhanced Learning in a Challenging Biology course. Winston‐Salem, NC: WSSU ‐ CETL (multimedia).

Overholt, J. L., & Galvis, A. H. (2010). Redesign of BIO 3311, Fundamentals of Anatomy and Physiology, and accompanying Lab (BIO 3111). Winston‐Salem, NC: WSSU ‐ CETL (multimedia).

Panopto. (2008). Case Study. Lecture 2.0: Suffolk University implements CourseCast to enrich tradicional and virtual classroom experience. Boston, MA: Suffolk University.

Schaffhauser, D. (2009, June 10). Campus Technology 2009. Retrieved July 03, 2010, from Lecture Capture Is Getting Campuses Talking: http://campustechnology.com/articles/2009/06/10/Lecture‐capture‐is‐getting‐campuses‐talking

Toppin, I. (2009). Student perceptions of the impact of using Video Lecture Capture. Winston‐Salem, NC: WSSU ‐ Center for Excellence in Teaching and Learning.

WSSU. (2010). Achieving academic distinction: The plan for student success. Winston‐Salem State University: WSSU, Office of the Chancellor.