Paradox in a Graduate Program Delivered On-Line

in a Human Science Curriculum

Janet Jeffrey RN, PhD & Mina Singh RN, PhD School of Nursing

York University Gail Lindsay RN, PhD School of Nursing UOIT

Toronto

Funded by the Office of the DeanFaculty of Health

May 6th - 2009 CASN Nursing Research Conference

PurposeMScN Program Evaluation Project

To conduct program evaluation of the new on-line MScN program that is participatory with both formative (improve the quality of the program) and summative (determine the worth of the program) components, to ensure accountability to all stakeholders.

To examine the design and delivery (process) of the MScN program as well as its outcomes ensuring depth as well as breadth given the proposed diverse data collection methods over time.

This presentation focuses on the first cohort of students in the MScN program (2005-2007) and the faculty who taught in the first year of the program.

Background

Masters in Science in Nursing (MScN) program was the first on-line graduate program at York University and in Ontario.

The first cohort of students was admitted May 2005 (after program approval by the Ontario Council on Graduate Studies in March 2005). So there was little time between application, admission, and starting the program for both faculty and students.

Although distance learning has been available for almost 80 years, research has focused on singular issues in online education such as examining and/or measuring students’ experiences in terms of what matters to their learning

Context

Philosophy of the MScN program is human science that values: Lived experience and Faculty in relationship with students.

Students are engaged in learning in an environment that is technology-based and at a distance.

We were concerned that the online method of program delivery was somewhat contradictory to the human science perspective which could be more easily delivered face-to-face and wondered how students would get connected and establish relationships with faculty.

Research QuestionsProcess

To what extent was the online MScN program implemented as developed/planned? What strengths and weaknesses of the program are identified by

students and faculty? How is the online delivery congruent with students’ approaches to

learning? In what ways are faculty-student relationships developed and

maintained in the online learning environment? How well do resources support student learning and faculty

development (Library, Centre for the Support of Teaching, Computer Help Desk & Computer Services)?

In what ways can implementation of the MScN program be improved?

Program Evaluation Design

Qualitative approach- focus groups and journal writing by both

students and faculty during the program

Quantitative approach - quasi-experimental design with

questionnaire data collected at the time of students’ entry to the program and after graduation by both students and faculty (students and faculty serve as their own controls)

Sample

Sample comprised of three groups:

1. Students admitted to the MScN Program to begin in May 2005.

2. Faculty who are teaching the courses offered to this first cohort of students.

3. Other stakeholders.

Data Collection - Students

Entry During Courses End of Each Course After Graduation

Questionnaires• Study

Processes Questionnaire

• Edmonton Research Orientation Survey (EROS)

Journals (1 minute papers)

Course Evaluation

Interviews

Focus group meeting beginning of each semester

Questionnaires• Study Processes

Questionnaire• EROS

Course Assignments

Students: Sample Size

2005 2006 2007

Full-time* In program Participated in evaluation

8

5

4

3

Part-time** Admitted Participated in evaluation

20

16

13

8

13

8

* graduated Fall 2006 ** 7 graduated Fall 2007

All students admitted to begin the program in May 2005 were invited to participate in the study

Some students did not really start the program; others left for academic and/or personal reasons

Students: Description

21 students originally agreed to participate in the evaluation 11 completed the evaluation of 28 of the first student cohort

who completed their MScN (10 females and 1 male) average age 45 years (SD = 6 years; range 35-55) All but two completed their baccalaureate nursing

degrees from 2003 to 2005; the other two in 1978, 1998Average number of years of full-time work experience

19.45 (SD 9.4, range 8-32 years) primarily as clinicians3 working part-time worked average of 9.3 years (SD

= 6.1, range 4-16 years)

Students: Questionnaires

Study Processes Questionnaire examines preferred, ongoing and contextual approaches to learning; deep and surface strategies describe the way students engage in tasks

• Deep approach M = 40.7 (SD 5.4) increased slightly (M = 42.7, SD = 3.1; NSS)

– Did the program attract these type of learners?• Surface approach M = 16.0 (SD = 3.5) increased

slightly (M = 18.0, SC = 7.8; NSS)

Biggs, J., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149.

Students: Questionnaires

Edmonton Research Orientation Survey measures overall research orientation, how research is valued and strategies undertaken to promote/use research

• For 5 students who completed EROS at both times, Mean of 38 items prior to starting the program M = 2.78 (SD = 1.3) and at graduation M = 2.75 (SD = 1.0)

• For all 11 who completed EROS at baseline M = 1.61

Pain, K., Hagler, P. & Warren, S. (1996). Development of an instrument to evaluate the research orientation of clinical professionals. Canadian Journal of Rehabilitation, 9(2), 93-100.

Students: Questionnaires

Course evaluation provided information that was similar to what we learned from students in focus groups• Online Learner Support Instrument (Atack, 2001)

• Interaction with peers and teachers• Course design and resources• Technology and environment

• Perceptions of Learning Environments Questionnaire (QUT, 1994)

• What helped and hindered learning formatted for each course

Attack, L. (2001). Web-based continuing education for registered nurses: Clinical application and learners' experiences, unpublished doctoral dissertation, University of Calgary, Alberta, 2001.

Queensland University of Technology (QUT). (1994). The teaching and learning in tertiary education (T&LiTE) project. A report prepared for the Teaching and Learning Committee, Queensland University of Technology, Australia.

Data Collection - Faculty

Entry During Courses/Program Delivery First Graduation

QuestionnaireApproaches to Teaching Inventory

Journals (1 minute papers)

Focus group meeting monthly during semester

QuestionnaireApproaches to Teaching Inventory

All faculty teaching the first full-time cohort of students were invited to participate in the study - teachers of the required courses, electives, and major research projects.

Almost 90% are participating.

Faculty: Questionnaires

Approaches to Teaching Inventory) measures ways in which teachers approach their teaching

• Conceptual change -- student-focused• Prior to teaching M = 32.0 (SD = 3.6) stayed stable

over the first year of the program

• Information transmission -- teacher-focused• Prior to teaching M = 13.5 (SD = 3.1) unexpectedly

increased M = 15.8 (SD = 2.9; p = .018)

Prosser M., & Trigwell K. (1999). Understanding learning and teaching: the experience in higher education. Buckingham: Open University Press.

Student Perspectives on Achievements, Challenges, and

Recommendations

Analysis of journal writing and focus group meetings lead to list of achievements, challenges, and recommendations.

Student Perspectives:Achievements Beginning the

MScN Program

Getting started (getting course readings/text, familiarity with WebCT, powerpoint)

Getting organized Completing academic work on-time, meeting deadlines (work

and school) Developing sense of community with other students in the

course – to provide support Balancing demands of work, family, and student roles Remaining in the program over time, “Hung in there until the

end” Obtaining “good” grade(s)

Student Perspectives:Challenges Beginning the

MScN Program

Time management, keeping up with multiple responsibilities Navigating the system at York; inconsistent access to virtual lab and on-

line library Silence of peers & teachers Mastering group conflict Learning the language of human science Fitting in and being seen as worthy within the program Sacrificing others areas of life to complete academic work

“one more paper and Mommy will be done” Fear of posting on-line Emotional toll – depression, anxiety, fear Embodied toll (somatic complaints) e.g., carpal tunnel syndrome, arm

tingling

Student Perspectives:Achievements Continuing into

2nd Semester

Logistics of starting courses – getting readings, texts etc

Pacing, keeping up in course work Staying on top of timeline/deadlines Balance home/work/school Staying motivated to participate

in the course

Student AchievementsDifferences from Entry to Program

“Made it through the summer”, “Survived, “Still going strong” Persevering

Student Perspectives:Challenges Continuing into

2nd Semester

Expectations for participation on on-line discussions (different for each course)

Lack of support and feedback

Timeliness of posting from peers in response to assignments

Student Challenges Differences from Entry to Program

Content of second term like learning new language (in new course)

Keeping peace and harmony within group, competition and hostility within group

Balance

Student Achievements1 Year Later

Students supporting and helping each other Building of student network

- Online

- Phone

- Workplace Persevering through the storm

- “New love for research is probably my greatest achievement..

- “ Always finishing the course”

- “Teaching sessional position”

Student Challenges 1 Year Later

Not enough time between courses Very tired, back to back semesters intense Workload heavy Accessing virtual software problematic Course selection restrictive Course objectives and assignments

still disjointed Issues re use of preceptors

Lessons Learned1 Year Later

Liked the philosophy of the program Convenient, accessible Review of courses PLEQ

• Helped- Presented on online symposium- Feedback obtained- Colleagues’ experiences

• Hinder- Too much theory - Online discussions were repetitious- Reading others’ notes- Others’ not contributing

The First Graduates Look Back

“More time, more time..”

“ Student input was valuable”

“Human to human close relationship is valuable”

“ Workload phenomenal”

“Learned….

Faculty Perspectives on Achievements, Challenges, and

Recommendations

Analysis of journal writing and focus group meetings lead to list of achievements, challenges, and recommendations.

Faculty Achievements

Program activated in 6 weeks Finding resources for learning online and course

development Establishing congruence between course processes,

content and learning outcomes Responding to students’ formative evaluations flexibly Surviving the workload Bringing our own research (content and method) to

curriculum

Faculty Challenges

No dedicated, timely, expert online education expertise designated Courses not designed with web expertise, multi-media rare Lack of peer consultation & feedback available Overall workload & finding uninterrupted time to work on course Context of MScN Program in School of Nursing How to see student engagement with the course and their learning Facilitating on-line small group conflict resolution Feeling disconnected, ambiguity about learning, ‘lost in space’

Lessons Learned

Preparing an online class is like slow motion Be prepared to draft, redraft, edit, rewrite, … Many faculty did arrange face to face meetings with students

at least once during the course Formalize CST, ACS, & FSC secondment/assignment to

MScN Program Obtain work study students to be web/media designers Concept map the program using the first course outlines to

evaluate coherence, development, etc. Explore shared formats for participation and other evaluation

processes Finding a balance between students’ expressed wish for

faculty participation Professional development for faculty

Mirroring …. Paradox

Faculty complain of feeling disconnected from students: Students complain about faculty availability.

Technology-enhanced learning is its own subject matter: Need for ongoing professional/faculty development and support.

At the start-up of each semester, some of the teachers are novice in the online environment: Students have cumulative expertise over time as their experience is continuous through the program.

Technology, while at a distance, is not disembodied: Faculty report tendonitis, eye strain, and fatigue as well as emotional stress.

Actions Taken Moving Forward

Students’ contribution to formative evaluation resulted in changes to

• orientation for incoming cohorts of students (now online)

• way in which WebCT platform is being used for course delivery; since changed to Moodle

Subsequent curriculum development undertaken:

• Concept map the program using the first course outlines to evaluate coherence, relationship to program goals

• Explore shared formats between courses for participation and other evaluation processes

• Consideration of how students work in groups on-line

Actions Taken Moving Forward

Improved access to York systems (virtual lab, library resources, learning how to use new software)

New option by university bookstore to order texts online for free delivery prior to courses starting (coincidental)

Development of two delivery options to match to students’ preferred way of learning

• on-campus (face-to-face) option – to start Fall 2009

• distance option fully online – no presence on campus

Conclusions & Implications

It is important to act on lessons learned formatively and summatively and to ensure continuous program evaluation. Evaluation has resulted ongoing shaping of the curriculum and how it is being delivered.

Ongoing evaluation of the purpose of technology philosophically & pedagogically must be undertaken to explore the paradoxes of implementing a human science curriculum in an online environment.

For Further InformationContact

Janet Jeffreyjjeffrey@yorku.ca

(416) 736-2100 ext 66696

School of Nursing, HNES York University4700 Keele St.,

Toronto, Ontario M3J 1P3Canada