final evaluation

Program Evaluation Plan

Arrupe College of Loyola University Chicago

Emily Ferron and Meghan Funk

Created Fall 2015

PROGRAM EVALUATION OF ARRUPE COLLEGE 2

Table of Contents

History of Arrupe College of Loyola University Chicago ……………………………………..3

Mentorship in Higher Education………………………………………………………………..4

Peer Mentorship Model at Loyola University Chicago………………………………………..5

Stakeholders………………………………………………………………………………...........6

Logic Model………………………………………………………………………………............8

Rationale for Evaluation……………………………………………………………………….12

Evaluation Approach……………………………………………………………………...........13

Quantitative Approach…………………………………………………………………………15

Qualitative Approach…………………………………………………………………....……..22

Limitations………………………………………………………………………………………28

Timeline………………………………………………………………………………….……...29

Budget…………………………………………………………………………………..….........29

Next Steps……………………………………………………………………………………….30

References…………………………………………………………………………………........ 32

Appendix A: Arrupe College Supporting Documents………………………………………..33

Appendix B: STARS Program Supporting Documents……………………………………...36

Appendix C: Logic Model for Mentoring……………………………………………………..37

Appendix D: Survey……………………………………………………………………….........38

Appendix E: Survey Construct Map……………………………………………………..........42

Appendix F: Focus Group Protocol…………………………………………………………...45

Appendix G: Consent Forms…………………………………………………………………..47

Appendix H: Email Invitations……………………………………………………………...…49

Appendix I: Timeline …………………………………………………………………………..50

Appendix J: Budget……………………………………………………………………….……51

Appendix K: Power Point Presentation………………………………………………….........52


History of Arrupe College of Loyola University Chicago

Arrupe College of Loyola University Chicago1 is a two-year newly developed Jesuit

community college centered in downtown Chicago. Arrupe College opened its doors in the fall

of 2015 with a population of 159 students. It was created to provide affordable higher education

to those who may not be able to access higher education without an affordable outlet. Arrupe

College is a structured, holistic learning experience for students who have recently graduated

from high school, located in the Chicago area. An additional goal of the college is that students

obtain an associate’s degree while graduating debt-free. Therefore, it is a hope that students will

work while in school in order to offset additional costs of education. (Arrupe College, 2015).

The student population is comprised of commuters, the majority of whom qualify for Pell

Grants and other state and national aid due to their financial eligibility. Currently there are

students who do not qualify for financial aid, including students who are undocumented and

those who can afford to pay for their education. The students receive a small scholarship from

the university and the college will help students find work-study and other positions on and off

campus. For the students who do not qualify for federal work-study, the College assists in

finding work and creating partnerships with stores and restaurants around the Water Tower

Campus. It is ideal that Arrupe student’s work 20-25 hours per week, which will help pay for

student fees throughout the semester. This class schedule and cohorts enable them to have a

healthy balance of school and work throughout the week. The cohorts also allow students to

have much smaller classroom sizes for a more efficient learning environment. The week is

divided into two cohorts, morning and afternoon, in which students study together and take class

1 Arrupe College of Loyola University Chicago will be referenced as Arrupe College


at the same designated times. Therefore, students who work can plan out their work schedules

around their class schedules.

In addition to the morning and afternoon cohorts, another interesting structure of the

College is that instead of operating on 15-week semesters, academic terms are divided into two

eight-week sessions where students take two classes twice a week, for three hours at a time.

Students attend school for 40 weeks per year, including the months of June and July. During the

semester breaks students are expected to be at workshop and training opportunities. These

include workshops for writing, resumes and cover letters, note taking, and other opportunities to

enhance their academic and work life.

Currently, because it is in its first year as an operating institution, Arrupe College has a

very small staff and faculty. The staff makeup is divided into Administration, Department of

Student Success, Department of Academic Affairs, Admissions, a licensed social worker and

faculty. Additionally, there are only six full time faculty members and three adjunct professors.

In addition to the aforementioned, there are limited resources pertaining to programming

and supportive services. Due to the limited amount of resources, Arrupe College shares many

resources with Loyola University of Chicago (LUC), such as tutoring support, academic

coaching, and other supportive academic outlets. However, there are not many supportive

programs used for the development of the students outside of the classroom that are unrelated to

academics. As a result, one of the major initiatives that Arrupe College would like to incorporate

is a peer mentorship program. Additional information regarding the College can be found in

Appendix A.

Mentorship in Higher Education


According to Whitman (1988), there is enormous power in peers to influence positive

learning and developmental growth. In the past, mentors have typically been graduate students,

faculty, and staff, but recently there are increasing formal roles for undergraduates to mentor and

at times teach their peers (Smith, MacGregor, Matthews, & Gabelnick 2004). According to

Terrion and Leonard (2007), it is now very common for institutions to match “mentors and

mentees who are roughly equal in age, experience, and power to provide task and psychosocial

support” (p. 150). This way the students share more commonalities and are more approachable

then if it was someone older or with more expertise. The peer mentorship also develops a higher

range of psychosocial development in which “confirmation, emotional support, personal

feedback, and friendship” are achieved (Terrion & Leonard, 2007; p. 150).

The peer mentor experience includes peer interactions with student pairs who share their

knowledge and experiences to help improve students’ understanding and learning. Additionally,

peer mentor roles help develop learning communities and environments for students to succeed

by offering additional support and connection (Terrion & Leonard, 2007). Peer mentorship

models have been used in a variety of ways in higher education institutions across the country.

There are various models in which these are used, specifically for first generation college

students, students of color, undocumented students, and so forth. For the purpose of this

evaluation, the evaluators will be modeling the logic model after other peer mentorship

programing at LUC. The evaluation is to determine if Arrupe students could benefit from a

mentorship program, and if so, what specific program traits could be most useful.

Peer Mentorship Model at Loyola University Chicago

Currently, there is no programming for peer mentorship at Arrupe College; therefore, for

we will be utilizing LUC’s Students Together Are Reaching Success (STARS) to inform our


evaluator and create a logic model on the current needs based assessment (Loyola University

Chicago, 2015). Information regarding the STARS mentorship program can be found in

Appendix B. The evaluators believe using this program to inform our evaluation; we will be

able to construct a logic model and questions that will help in the development of a mentorship

program. The STARS program is a peer mentorship program, which helps first year students

connect with upperclassmen for one-on-one and/or group mentoring (STARS Program, 2015).

With this program, students are a not only involved in peer mentorship, they are also able to

attend social and cultural programs, participate in workshops in order to increase academic

success, and enhance their personal understanding and growth in leadership workshops. This

mentorship program is housed in Student Diversity and Multicultural Affairs (SDMA) and is

open to students of color and/or first generation college students.

The STARS program is specifically designed for their mentees to meet other first years

and gain understanding and use resources to reach college success. They hold monthly events

focused in social events and educational opportunities where they find support for their

academics. Monthly events range from eating lunch together, attending events around Chicago,

and taking in cultural activities. STARS mentors also attend other workshops and retreats in

order to develop professional skills and their own values more deeply. Furthermore, STARS

mentors have the opportunity to meet alumni from the program to enhance their sense of

community and networking.

Stakeholders

Arrupe College is a brand new institution and within its first year of operation, therefore,

there are many stakeholders who would find peer mentorship to be very important and critical to

the development and retention of the students. According to Wholey, Hatry, and Newcomer


(2010) stakeholders are “individuals, groups, or organizations that can affect or are affected by

an evaluation process or its findings” (p. 31). The primary stakeholders are the students, staff

and faculty, and Board of Trustees of Arrupe College. Arrupe College faculty and staff work

directly with the students and would be involved in the implementation and execution of a

mentorship program if it is determined one is needed.

The primary stakeholders of a peer mentorship program at Arrupe College would be the

students, staff, and faculty on campus. The faculty and staff report to the Dean of Students and

work together to ensure the success of the students. Therefore, these stakeholders are on the

front lines of student development and success and therefore primary stakeholders for this

evaluation. Wholey et al. (2010) described stakeholders as those who are invested through

various forms of power, which include authority, legitimacy, financially, and in other ways they

influence individuals and groups. Further, having students invested in programming outside of

the academic arena will help foster higher retention rates due to their personal investment at the

institution. As a result, administrators, such as the Dean of Students, Dean of Student Success,

and Dean for Academic Affairs, will all be major stakeholders in this evaluation.

Furthermore, because there are many shared resources, the Loyola community is in many

ways an indirect stakeholder. There are many ways students and departments are able to

collaborate or develop programming accessible to Arrupe College students. As Arrupe students

because more invested in the college, they are able to participate in additional programming at

the LUC campus. If students are succeeding, they may become more invested in other activities

at LUC and within the community. Additionally, with the current partnership and shared

resources with LUC, staff members and administrators at LUC will also be important

stakeholders in this evaluation. For example, the STARS program in housed in SDMA;


therefore, SDMA will be an important resource for implementing the mentorship program with

current college students.

Lastly, another stakeholder is the greater community of Chicago who will benefit from

more talent in the community. Having more low-income students having access to affordable

education and having a mentorship program, which fosters growth and commitment to that

advanced education, will be benefit from this program.

Logic Model

The follow is a narrative of the Logic Model. See Appendix C for model. This model is

adapted from the STARS peer-mentorship model for Arrupe College students.

Resources

In order to have successful outcomes a number of resources will be important.

According to Wholey et al. (2010) resources can be “human and financial resources as well as

other inputs required to support the program” (p. 57). It is important to categorize these

resources as space, people, and budget. To begin, it is important there is space for the

mentorship program to meet at Arrupe College. The peer mentorship groups will need a place

and a space to meet on or off campus and it is important those spaces exists and are available.

Additionally, the main people who will be used as resources will be the advisor of the

program, whether it be a staff person or graduate assistant who will be coordinating this

program. There will also be staff, faculty, and peer mentors involved in this mentorship program

who will be participating in the functioning of the program. Individuals that will be involved are

the Deans of both the Department of Student Success and Academic Affairs because they will

oversee the program and its facilitation. Other individuals that will be important are the campus

partners who are the facilitators of the academic and leadership workshops.


The budget will also be an important factor for Arrupe College and will affect what

students can do outside of the mentor/mentee relationship. It is important that students develop a

relationship outside of the classroom. This program will provide snacks, soda, and other

resources on various occasions to be an incentive for students to participate. Additionally, there

may be a cost for social or cultural programs as well as compensation for the facilitators of the

workshops.

Outputs

Wholey et al. (2010) described outputs as the “products, goods, and services provided to

the program’s direct customers or program participants” (p. 57). The mentorship program is

designed for students who are struggle either in or outside of the classroom or a combination of

both. In the mentorship program, students will engage in both one-on-one and group mentorship

in order to foster a peer relationship, but also grow academically. Additionally, students will be

able to participate in social and cultural events, leadership workshops, and academic workshops.

Outcomes and Impact

Wholey et al. (2010) defined outcomes as “changes or benefits to people, organizations,

or other program targets that are expected to result from their being exposed to activities or

outputs” (p. 58). Outcomes are divided into three categories, short-term, medium-term, and

long-term outcome, where many outcomes will overlap with two or more areas.

Short-Term Outcomes

There are many short-term outcomes appear if a mentorship program is put into place at

the College. Our first outcome is that students are able to talk about their identities and how it

may or may not relate to their identity as a student at Arrupe College. There is high percentage of

students who identify as people of color and first generation at the College and it would be


helpful for students to have a space that supports these conversations. Second, as a result of peer

mentoring, students will begin to develop a sense of belonging to the university, which could

increase their rate of retention. Students will also develop a peer group and a network of support

at the university. If students are connected to other students it will help establish a rationale for

students to stay at the university. It is important that students succeed not only academically but

also socially while in college and have a sense of community and belonging. Third, it is a hope

that students begin to succeed at higher levels inside of the classroom. Having a connection to

someone outside of the classroom may also help students be motivated to connect to the

classroom. Additionally, we believe that this program will not only help increase retention

academically, but will create an investment for the students and a loyalty to the College.

Medium-Term Outcomes

There are multiple medium-term outcomes that may result from student’s involvement in

a mentorship program at the College. First, students’ academic performance in the classroom

should increase over time if they are participants in the mentorship program. It is a hope from

the evaluators that if students have a support group at the institution, they will want to stay and

perform better in the classroom. Additionally, by providing workshops and resources to increase

academic performance, it is a hope that students will perform at higher levels. Our next outcome

in linked to academics, in that students maintain a 2.5 grade point average (GPA) or higher. As

evaluators we hope that students will perform better in the classroom and continue to be in good

standing or higher. Third, students will deepen previous relationships and build on connections

from the previous semester and/or academic year. It is the hopes of the evaluators that this will

continue their commitment to the College and their relationships and over time gain deeper

perspectives and relationships with each other and the College community. If students become


more invested with their peer group, receive workshops and skills on how to be a better student

and leader, and students perform better at the university, it is assumed that students will stay

loyal to the university. Fourth, students develop a deeper level of self-confidence. It is the hopes

of the evaluators that students will build self-confidence because they are more comfortable with

themselves and others because of their connection to their peer group. Next, students are more

comfortable and connected to the college community. By becoming more comfortable and

connected to peers and themselves, students will be more comfortable being at the university.

This also develops a sense of Loyola to the college which will tie into our next outcome. Our

last medium-term outcome is that students continue with their studies at the College. It is our

hopes for retention that students stay at Arrupe not only for academic reasons, but also for their

social life and loyalty. Since students are more comfortable at the university it is a hope from the

evaluators that students will want to stay at the university and continue their students.

Long-Term Outcomes

There are many long-term outcomes that may be attributed to a mentoring program.

First, it increases loyalty to the university and allows students to feel connected to the College

where they stay and continue to feel invested. Similar to the short and medium term outcomes,

the evaluators also believe a mentorship program will attribute to better student outcomes and

even allow students to be better leaders. Second, students’ efficacy and capacity for leadership in

multiple spheres will increase over time. When students receive skills through the workshop and

are connected to their peers, they may receive an increased amount of efficacy. Third, students

will graduate from the College. If students succeed in the classroom, develop a peer group, and

stay at the university, graduating from the university is much more likely. Fourth, students will

move consider attending a four-year institution upon graduation. It is the mission of the College


to assist students in graduating from a two-year institution and transition toward a four-year

degree. The evaluators believe the mentorship program will assist students in succeeding in the

classroom and building efficacy in order to achieve success at a four-year institution after

graduating from Arrupe College. It is important to recognize with all of these projected

outcomes, there are still many assumptions and external factors that may affect the evaluation.

Assumptions and External Factors

While designing the logic model and planning for an evaluation of a mentorship program

at Arrupe College, the evaluators considered several assumptions. First, the evaluators assume

the mentor and mentee relationship will be a positive one. It is the evaluator’s hope that students

participating in the program are matched properly, but one cannot be sure that it is a healthy

mentorship relationship. Second, the workshops to increase academic and leadership skills are

assumed to be helpful and well attended. Third, the evaluators assume the topics of the

workshop will be relevant to what students actually need. Fourth, there is a staff member or

graduate assistant that will be able to run this program. The fifth assumption is that students

involved in the program will be comfortable with the mentors and staff members involved in the

program.

In addition to the assumptions, the evaluators were also cognizant of the external factors

that could affect the mentorship program. First, that the student will attend their mentorship

meetings and the workshops. Second, that timing and locations where programs and mentoring

will take place will be available to our students. Third, students will have time in their schedules

to be a part of the program. Due to the model of the program, student availability looks

differently than other institutions and therefore, they may not be available during the times the


program is offered. These assumptions and external factors will be important to assess during

the evaluation process in determining the best fit for a mentorship program.

Rationale for Evaluation

There has been no formal evaluation completed for programming at Arrupe College thus

far. When assessing what programs Arrupe College needed, it was evident that a mentorship

program was an important aspect the administration wanted to implement. According to Cramer

and Prentice-Dunn (2007), “freshmen assigned to mentors show greater gains in problem

solving, goal setting, and decision making compared to their non-mentored counterparts” (p.

771). In trying to provide Arrupe College students with the best opportunities for academic

success, it is important to conduct a needs assessment to see if students at Arrupe College would

positively benefit from a mentorship program within their institution. If students indicate they

would benefit from a mentorship program, it is also important to determine what elements of a

mentorship program will best support Arrupe College students. The evaluators believe this

assessment will create the foundation for a new program that can be assessed and improved for

future use. By assessing current students’ needs, along with input from current faculty and staff,

it is the evaluator’s expectation to build a supportive environment for Arrupe College students to

be successful both in and outside of the classroom environment. It is important when designing

an evaluation that one understands the best and most effective evaluation approach to utilize

(Fitzpatick, Sanders, & Worthen, 2011).

Evaluation Approach

One of the primary goals of Arrupe College is to provide students with an environment in

which students can achieve academic success. As a result, the evaluators will conduct a

formative evaluation of a peer mentorship program at Arrupe College. Fitzpatick et al. (2011)


stated “an evaluation is considered to be formative if the primary purpose is to provide

information for program improvement” (p. 20). Arrupe College does not currently have any peer

mentorship programming; therefore, the evaluators’ evaluation approach will consist of a needs-

based assessment. The evaluators will investigate and assess any specific needs of the students

as they pertain to a peer mentorship program and academic success. The evaluators will assess

students’ needs by utilizing both quantitative and qualitative measurements of students’

experiences. There are various strengths and weaknesses for using the needs-based assessment

model for the evaluation.

There are multiple strengths in using this model, beginning with the development of a

new peer mentorship program based on the needs of current students. This institution is also

newly designed with a specific and unique purpose; consequently there is freedom to design a

new and innovative model for peer mentorship that will specifically complement the current

student population. Furthermore, many of the stakeholders are approaching peer mentorship

with an open mind, where students, staff, and faculty are able to voice their needs and they are

not based on previous models of peer mentorship, which are currently in place.

There are a few weaknesses of using this type of assessment. The main issue, which the

evaluators are facing in this assessment, is that Arrupe College is a brand new institution and

therefore, there is no model in order to base this specific issue. There is no internal research

since this model has not been developed at this institution and for these students in particular. As

formally stated, the students of Arrupe College are unique to this newly developed university

structure and there is an additional lack of information on this specific population. Another

challenge is we are only evaluating this specific cohort of individuals; therefore the data we may

gather may not be generalizable to the greater Arrupe community in the future. Aside from the


aforementioned, this study will not show trends or outcomes of the peer mentorship program.

The evaluators will be setting the foundation for a peer mentorship program, and will not be able

to assess process and outcomes at particular point in time.

Specifically, the evaluation will focus on the needs of the student population as they

pertain to peer mentorship. The staff and faculty will be utilized in ways they see students need

mentorship in regards to academics, social behavior, support for diversity, and development of

life skills and professional development. It is hopes of the evaluators that this assessment will

provide feedback for Arrupe College in order to build a base for a transformative peer

mentorship program. It is important to investigate and assess peer mentorship in order to adhere

to the mission of Arrupe College in which student success and retention is effectively met. It is

the hopes of the evaluators that a sense of belonging on behalf of the students will increase

retention based on the development of a peer mentorship program.

Quantitative Instrument

There is a belief that mentorship programs contribute to the transition of the new student

into their first semester of college (Smith, MacGregor, Matthews, & Gabelnick 2004; Terrion &

Leonard, 2007; Whitman, 1988). The primary factor for engaging in a needs-based evaluation of

a mentorship program at Arrupe College was because of the lack of current programing and the

potential for an increase in mentorship based on needs expressed by students. This quantitative

analysis will provide information about the needs of the students in regard to mentorship and the

type of mentorship they may find most beneficial. Example mentorship programs include peer

to peer, Loyola student to Arrupe student, staff and/or faculty to student, and professional in the

field to student. Additionally, the evaluators have developed differing programing, resources,


and considerations that may be beneficial. This selection will discuss the quantitative design,

survey design, and statistical analysis created for this evaluation.

Population

Based on the purpose of the evaluation, the evaluation will focus on the population of the

College students during the after the first eight week term of the program. If this was to be

repeated in future years, the evaluators would suggest using the population of first year students.

There are currently 159 students enrolled at the College. The evaluators chose a census sample

because it allows for anyone in the community to participate and because it consists of a small

group size (Wholey et al., 2010). The survey will be conducted at the mandatory Town Hall

meeting, which happens at the conclusion of the first eight week session. Allowing students to

complete the survey will ensure no additional cost will be acquired and there will be close to full

participation (Wholey et al., 2010). Since some students still may be unable to participate due to

illness or other commitments, evaluators will email the survey link to absent participants in order

to gain participation from the missing students. This survey link will be open for two weeks

during which, evaluators will send the initial email containing the survey link. Evaluators will

then send two additional email reminders, one as a one week reminder and the second on the

final day to remind participates to complete the survey before the link closes.

Research Design

Evaluators have decided to utilize a cross-sectional, pre-experimental design because the

participants will not be randomly assigned or selected (Creswell, 2009). Additionally, there will

be no comparison or control group because all participants will be the College students and the

treatment is not being assessed. Additionally, this will not be a longitudinal study because it is

solely assessing the current needs for a mentorship program at the College.


A cross-sectional design has much strength, which will be very beneficial in this study.

First and foremost, it allows for the evaluators to compare multiple groups within the population

at the College very easily. Additionally using the cross-sectional design will provide an

assessment for the current needs of the College. It is also helpful that the cross-sectional design

will be more cost-effective then the longitudinal study. Furthermore, there also seems to be a

higher participation if students participate one-time instead of over a period of time.

Though there are many benefits of the cross-sectional study, there are also limitations.

This type of design does not allow for causal inferences. For our specific study we are not

seeking to provide causation, only to discover the current needs of the program. This design is

specifically used to gather data at one point in time. Due to the nature of this study, the

evaluators will not be seeking process, but will be using some outcomes-based data. The

outcomes-based data will be in the form of information pertaining to students’ experiences

during their first semester at the College. Evaluators believe the data will be very intriguing for

our needs-based study.

Survey Instrument

The main survey instrument that will be used in this study will be a web-based survey,

which will evaluate the needs of a mentorship program at the College (Appendix D). The web-

based survey should take no more than ten minutes. Schuh (2009) described the need to simplify

the survey and replace long words with shorter synonyms. The evaluators took this into account

while developing the survey and tried to use vocabulary and phrases with shorter synonyms so

students better understand the terminology.

As in the aforementioned, Appendix D shows the survey that the participants will be

receiving. Appendix E describes the map of connections, which the evaluators made between


each item and the outcome to demonstrate the validity of our surveys (Schuh, 2009). Questions

have been organized into groups based on the type of responses they were eliciting.

As evaluators, we intentionally started the survey with questions that will help them

reflect on their learning experiences and connection to the College thus far, both in and outside

of the classroom. Such as what academics have been like for them, how they have been

socializing, and their overall connections to people in and outside of the classroom.

The second section of questions focus on the student interest levels around engaging in

specific Arrupe and community opportunities. This section will help evaluators determine the

interest level of participants and getting more involved in campus initiatives, including a

mentorship program. As a needs based assessment, this section will be very important in

determining the interest level of students pertaining to additional programming at the College.

The third part of the survey contains questions more fitted to evaluating the needs of a

mentorship program. These questions take into account what form of mentorship is best fitting

for students at this institution. As previously described, this may take into account who students

are seeking as mentors, how often they would like to meet with a mentor, and what activities

would the mentorship provide for the students. Since students may have varying definitions of

mentorship, evaluators will provide a definition of mentorship on the survey prior to these

questions.

The final part of the survey asks participants demographic information. Questions in this

section ask the participants about racial identity, ethnic identity, grade point average, and current

year at the College. The evaluators included the demographic information at the end of the

survey because we did not want it to affect the participant’s responses in the first parts of the

survey. This data can also be analyzed by race, ethnicity, and GPA.


The evaluators have created the survey using Google Forms because of funding and time.

We have decided because of funding purposes, we will use a free platform in order to complete

our survey rather than pay for the services. We also intend to use Campus Labs, which is

Loyola’s assessment program as a way to administer the survey. To ensure that all participants

have access to a computer in order to complete the survey, the final Town Hall meeting will be

held in a computer lab located on the College’s campus.

We anticipate a very high response rate because the survey will be completed before

classes for the second eight week session begin. It is the hopes there will be an 85% completion

rate, knowing some students will not attend the Town Hall meeting. Additionally, one would

infer some students may not complete the entire survey or honestly. As stated earlier, evaluators

will be sending a web link with the Campus Labs survey to students who were unable to attend

the meeting in order to increase the response rate.

Pilot Survey

We will use the Student Government to pilot test the survey. Many of these students are

familiar with the development of what is happening at the College and are familiar with a need

for a mentorship program. Many of these students are aware of the needs and struggles of the

student at the College and would benefit from pilot testing.

The pilot test will be conducted during a Student Government meeting, which happens

weekly at the College. The evaluators will be sure the students bring their laptops to this meeting

so they are able to take the survey. If students do not have their laptops with them, we will use

the student computer lap to complete the survey. After the survey is completed, we will ask for

feedback regarding the time of the survey, clarity of questions, and response scales.

Survey Administration


The survey will be administered via the internet. The evaluators chose this method

because we believe it will reach the most people in one sitting. The evaluators believe this

method will also help us analyze our data because it will be much simpler then manually entering

our data. The web-based survey will allow cut down on the amount of time it also takes to

analyze it because we will not to enter it manually. For many programs it may be dangerous to

use web-based surveys because not all participants may complete it, however the evaluators

believe if it is administrated all at once with all of our students in the same location, participants

will be compelled to complete the entire survey.

Schuh (2009) described incentives for taking the survey as both financial and

psychologically based. In survey collection it is also important to provide for a high response

rate. As an incentive for our program we will provide participants the option of entering into a

drawing for free Arrupe College gear from the campus bookstore. In addition, it is our hopes

there will also be a psychological incentive to this survey. Students will have time to review

their first eight week session and really identify their needs as students moving forward.

To maintain confidentiality, the final page of the survey will be where there is a link to

enter their name, email address, and phone number for the drawing. Though all of our students

will take this survey at the same time, while all in one room, it is the hopes of the evaluators that

this will provide incentive to complete the survey more efficiently.

The evaluators have chosen for the survey to be delivered during the first Town Hall

meeting after the first eight week session. We have chosen this time frame because students will

have completed their first eight weeks and grades will have been received for each of their

classes. At this point, students will be more aware of themselves as college students and their

potential needs moving forward. Additionally, students struggling academically will be more


aware of their own struggles, while students who are excelling will be more aware of gifts they

may have to offer their classmates. It is the hopes of the evaluators that students will better

understand their current standing and will be more aware of needing a mentor or becoming a

mentor in some fashion. In addition to academics, students will better understand their social

roles as they pertain to Arrupe College and LUC. The primary contact for the survey is the

Student Success Intern who will explain the importance of the survey during the Town Hall

meeting and attend to any questions students may have during the 10-minute period of

administering the survey.

Statistical Analysis

All of the data will be run through SPSS software to check for statistical analysis and

other important factors. The evaluators will run multiple data analysis, beginning with

descriptive statistics. According to Wholey et al (2010) descriptive statistics are used to

“describe a group of items” (p. 455). The evaluators would like to evaluate a simple census of

our students, which is why we have chosen this method. Additionally, we do not need to

generalize the results of the population because all of the students will be serving as participants.

Therefore we will not need to conduct inferential statistics (Wholey et al., 2010). The evaluators

have also chosen Likert-scale data because Wholey et al. (2010) argued it helps the present the

percentages of participants who chose individual ordinal points. In order to evaluate our Likert-

scale measures, we will be utilizing univariate statistics. We will test the data set for information

about frequency and central tendencies, including mean, median, and mode. We will use pie

charts and line graphs to display the data of frequency and percentages for each variable for the

most significant responses.


In addition to the aforementioned, the evaluators will also run categorical analyses to see

the percentage of respondents that check the types of mentorship programming they may need.

Based on other well-run mentorship programs we have elected to provide students with the

options of the College peer mentorship, Loyola peer mentorship, and/or mentorship by a staff,

faculty, or professional. Additionally, we have also provided students multiple resources they

will utilize during the mentorship program, including professional development, cultural

resources, academic workshops, and social events. All of this will be run similarly and we will

provide pie charts and bar graphs for both type of mentorship and resources needed.

We also want to evaluate the student’s current standing and their need for a mentorship

program. In order to assess and compare different group variables across various Likert-scale

questions we will be using the compares means function to see mean differences in the Likert

items across different grouping variables such as demographics and GPA. Next, we will run

Chi-square tests to compare different Likert-items such as questions pertaining to academics,

social belonging, and other forms of involvement at the College. This way we can base the

Likert-scale questions off of many different dimensions of the group and see where the needs of

the entire student population. These results will be represented on a cross-tabulation table, which

will provide a table of the multivariate frequency distribution of our variables.

Included in our evaluation, we will also be providing an executive summary of the survey

results. This summary will elaborate on what the needs of students are in terms of a mentorship

program. By doing this the evaluators will be able to provide recommendations to the program

for the following semester or academic year.

Qualitative Approach


For the qualitative approach, the evaluators decided to implement focus groups. The

focus of our evaluation is a need-based assessment and therefore many of our questions will

focus around the needs for a mentorship program for our students. The main goal of focus

groups is to “elicit ideas, attitudes, feelings, or perceptions about a particular topic” (Schuh,

2009; p. 88). The evaluators will utilize the focus group to assess what students would like to see

in a mentorship program and what the needs are for the students. According the Schuh (2009),

focus groups help provide more depth to the assessment process.

Paired with the quantitative instrument, this focus group will give students at the College

a voice to share how they learn or socialize and how a mentorship program will impact them.

The evaluators have chosen this instrument for many reasons. First, since the College is in its

first year of operation, we want participants to have a voice in its future. This focus group will

provide them an opportunity to share insights into their own development and how they may rely

on a mentorship program in the future. Second, the College is small and will continue to be a

small population; therefore, we want to develop a sense of community for the students. We hope

this will continue to develop a strong, intimate sense of community. This focus group will also

complement and add depth to the survey given to them at an earlier occasion.

Sampling

Our focus group will consist of students from the College. It is suggested that this be

implemented during the first year of the College and therefore will consist of first year students.

However, if this is conducted after the second year, the focus group will consist of first and

second year students. We plan to use stratified purposeful sampling method for our focus group

so we are able to select specific groups of students (Schuh, 2009). We will conduct two focus

groups for two different groups of students. First, we hope to have a focus group for students


who are excelling in academics and second, students who need additional help in their

academics. The evaluators believe that sampling these two groups will be beneficial in

understanding the two extremes of our population. By understanding more deeply each group,

we hope to provide feedback on both desired needs that would be beneficial for both groups.

We also hope to use simple random sampling with the class roster to determine the 6-8

students who will be invited into the focus group. Wholey et al. (2010) explained, “smaller

groups of five to eight are recommended for topics that might be seen as sensitive or personal”

(p. 382). We will invite students who we believe represent the College, in order to ensure

representation across important areas related to academic pre-major, race, gender, ect. If

students decline, we will continue to use random selection until we meet our number of

participant goals.

Protocol

Protocol for the focus group (see Appendix F) will be used as our guide to structure the

dialogue for the participants. The focus group will be slated for 90 minutes in length. The

purpose of the questions will be to gather information regarding the students’ interest and need

for a mentorship program at the College. We have divided the protocol into sections moving

from low-risk questions to more high-risk questions.

The first section of the protocol is the introduction. In the introduction one of the

facilitators will ask participants to introduce themselves, which cohort they are in, and what they

have selected as their pre-major. It is the hopes of the evaluators that these questions will help

participants enter into the conversation and enable them to get to know each other if they have

not already.


The introduction will also contain several housekeeping announcement, in which the

evaluators will provide the participants with the format of the focus group such as length of time,

the use of an audio recording, and the information around confidentiality. This will also be the

time consent forms will be distributed and signed by participants (see Appendix G). Evaluators

will then instruct participants to help themselves to the food provided and then the introductions

of participants will begin. Since the College has a population of 159, there is a high probability

that the participants will already be acquainted. This may pose as an advantage since the

students may feel comfortable sharing information in the focus group with their peers compared

to a focus group with strangers.

After the participants have introduced themselves, the evaluators hope the participants

will feel more comfortable transitioning into the second section of the protocol which includes

questions around one’s experience at the College, specifically in regards to academics, social

interactions, and support. Facilitators will then transition into portion three of the protocol by

asking participants questions around mentorship. These questions focus on different forms of

mentorship such as peer, Loyola student, faculty, staff, and professional mentorship. The goal

for evaluators is to uncover the specific mentorship needs of participants.

Evaluators will then end the focus group with the final protocol section by asking for any

final thoughts. By asking questions around additional needs of the participants, evaluators hope

to gather information they did not originally identify as possible academic support. Once

participants have finished sharing, evaluators will conclude the focus group by thanking the

participants and providing them with the evaluators’ contact information for any additional

questions or comments the participants may have prior to leaving the focus group.

Implementation


The graduate assistant at the College will facilitate the focus groups. To ensure this

individual does not unduly influence our participants from speaking authentically during the

focus group, we asked someone with less influence or authority in order to provide an authentic

space about mentorship. These individuals are good people to facilitate because participants are

familiar with them and they know more about the College compared to an outside source. These

facilitators do have an existing relationship with the students and we believe they would still be

effective facilitators. The focus group will be recorded via audiotape by the facilitators to best

capture what participants are sharing and this will be transcribed by a work-study student.

Participants who are selected will be sent an email in mid-January inviting them to

participate in the focus group (Appendix H). This invitation will be sent purposefully by

stratifying by academic progress. The email invitation will explain why they have been selected

and the purpose of the focus group. The invitation will allow students to RSVP via email and fill

out a survey that will indicate dates and times they are able to meet for the focus group. It is the

hopes that the focus group will be conducted by the mid-March so the information will be used

for the next session. We believe this is a good time to host a focus group because students will

have completed their first session and will hopefully know more of their needs as a college

student.

The focus group will take place in a classroom at the College so students have privacy

and it is a space that can accommodate six to eight participants and the facilitators. The focus

group will take place in the evening after the afternoon cohort is finished. Dinner will be

provided as an incentive to participate. Staff or faculty who will be our facilitators will be told in

advance the date and time so scheduling will pose no issue.

Data Analysis


As Wholey et al. (2010) suggested, the evaluators will use transcript-based analysis for

the qualitative part of the evaluation. We will have the Christ the King high school intern

transcribe the audio recording. During this approach we will use a grounded theory approach

(Crewell, 2009). This approach was chosen because this is a needs based study and the first time

the study is being conducted. Therefore, it will use an open coding system and will allow us to

determine the major themes. Many of these themes will be based off of what participants talk

about in the focus group. We are reluctant to use predetermined codes that could potentially

skew our data (Creswell, 2009). Additionally, this is the first time we are conducting this focus

group and do not have insight into what will be the potential codes.

The coding will be completed by both of the evaluators. Although one of the evaluators

works as an intern at the College and the other has no affiliation to the school, it is believed both

parties can remain unbiased in the process since this is a needs based assessment and no program

in currently in place. Therefore, the evaluators are conducting the qualitative assessment with a

primary focus on the needs of the participants.

As a precaution to refrain from biased opinions, both evaluators will examine the

transcribed audio recording separately. They will identify themes and note outlying answers as

to not dismiss these as valuable data. The evaluators will measure the frequency and

extensiveness of the themes (Wholey, 2010) in order to be specific in their findings. The

evaluators will then meet to discuss their individual findings in order to come to a final

conclusion in regards to the data presented from the focus group. In addition to the

aforementioned, the evaluators will also do member checking in order to help improve the

accuracy and creditability of our focus group and to ensure our own biases did not affect the

results.


A limitation to this evaluation is that both facilitators have a deep knowledge of

mentorship and mentorship programs. This knowledge may pose as a bias as evaluators attempt

to interpret the themes that arise within the focus groups. This limitation will need to be

addressed by the evaluators discussing and reviewing the goals of the evaluation and setting

additional goals in regards to how each will avoid biases as they pertain to mentorship programs

as well as our own biases in coding.

A final presentation will be provided to the College’s administration since they are the

key stakeholders in this assessment. The evaluators will present the data in the form of a

PowerPoint presentation, which will include charts in order to create a visual description of the

themes that arose during the focus group. Evaluators will also include quotes from the focus

group as supporting evidence in regards to the overarching themes. Evaluators will end their

presentation by making recommendations for a future mentorship program based on the themes

that arose from the focus group. Lastly, the evaluators will leave the administration with a

written document of the evaluators’ findings and recommendations for future use.

Limitations

The evaluators have put a lot of work into preparing this evaluation to produce accurate

and telling results. However, they can still identify several limitations that may be present. First,

being a needs based assessment combined with one of the evaluators working in the Department

of Student Success, can create a bias when pushing for a mentorship program. The evaluation

team has implemented several tactics to diminish this bias but utilizing student workers and

interns, but it is unlikely that all biases will be kept from this evaluation.

Another limitation is this assessment is using a population that consists of only one

cohort of students. It could be valuable for evaluators to assess both first and second year


students in order to identify a possible shift in the need for mentorship between a student’s first

and second year in the associate program. However, since this is the first year of the College’s

existence, it is not possible to have more than the one class that is currently enrolled.

A third limitation could be the lack of buy-in that comes from all the stakeholders

involved in this evaluation. This lack of buy-in comes from the institution being so new.

Students could have anxiety around succeeding at an institution that has not been in existence

before and has not developed a reputation. As a result, these students may not have put much

thought around their potential need for a mentorship program. The College faculty and staff may

also have anxiety around Arrupe being a new institution and feel a pressure to ensure the

institution is successful so they have job security. Therefore, faculty and staff have the potential

to overlook a mentorship needs assessment when prioritizing what they think is best for the

institution.

Logistics

As evaluators prepare to implement both quantitative and qualitative assessments, it is

vital to be aware of the logistics that go along with the process. In order to make this evaluation

efficient and financially sound, evaluators must construct a timeline and budget. The timeline

and budget can be located below in the following two sections, as well as in Appendices I and J.

Timeline

Since the College hopes to implement a mentorship program in the fall of 2016, this

evaluation will need to be completed in a timely manner during the spring 2016 term. As noted

in Appendix I, evaluators have created a timeline in order to keep the assessment on track. By

keeping to this timeline, evaluators will complete the entire evaluation in the course of six

months.


Budget

The budget for this assessment plan is $240 (see Appendix J). The evaluators are able to

have a smaller budget for this assessment since both the Department of Student Success and the

Department of Academic Affairs will be supplying access to their budget for many materials

needed. These materials include rooms on campus for the focus groups and Town Hall meeting,

computer for data storage and email, access to Campus Labs, and office supplies. The

Department of Student Success will also be supplying student workers and interns to diminish

the cost needed for focus group facilitators and transcribers. Evaluators will also have access to

Loyola’s Digital Media Lab to rent recorder at no cost for the focus group.

Although many resources needed for this assessment come at no cost to evaluators, there

will be a cost for the incentives used. Evaluators will be supplying Chipotle and soda at each

focus group. The cost of the food and drink being provided will be $95 per focus group and a

$50 gift card from the drawing for completing the survey. The evaluators expect this expense

will increase the number of focus group participants.

Next Steps

After completing this evaluation, evaluators intend to recommend the following next

steps to the College administration during the final presentation. First, since this is a needs based

assessment, it is important for the Department of Student Success and the Department of

Academic Affairs to continue to evaluate the mentorship program that is implemented. Also, the

student population is changing every two year, which could impact the types of mentorship the

students need. Therefore, implementing ongoing process evaluation can prove extremely

important. Another next step would be to monitor the future stakeholders at the College. Arrupe

is a brand new college that hope to continue to grow. Therefore, it is very possible that current


faculty and staff positions may be created and/or altered. As a result, it is vital that future

evaluators note the change in potential stakeholders and be open to altering who they bring into

future process evaluations when it comes to the mentorship program.


References

Arrupe College. (September, 10, 2015). Arrupe College. Retrieved from http://www.luc.edu/

arrupe/

Cramer, R. & Prentice-Dunn, S. (2007). Caring for the whole person: guidelines for advancing

undergraduate mentorship. College Student Journal, 41 (4), 771-778.

Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed method

approaches (4th Edition). Thousand Oaks, CA: Sage Publications

Fitzpatrick, J., Sanders, J., & Worthen, B. (2011). Program evaluation: Alternative approaches

and practical guidelines (4th Ed.) New York: Longman.

Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:

Jossey-Bass

Smith, B., MacGregor, J., Matthews, R., and Gabelnick, F. (2004). Learning Communities:

Reforming Undergraduate Education. San Francisco: Jossey-Bass.

Loyola University Chicago. (September 10, 2015). Students Together Are Reaching Success.

Retrieved from http://www.luc.edu/diversity/programs/stars/

Terrion, J. & Leonard, D. (2007). A taxonomy of the characteristics of student peer mentors in

higher education: Findings from a literature review. Mentoring & Tutoring: Partnership

in Learning, 15 (2), 149-164.

Whitman, N. (1988). Peer Teaching: To Teach Is To Learn Twice. ASHE-ERIC Higher

Education Report No. 4, 1988. ASHE-ERIC Higher Education Reports, The George

Washington University, One Dupont Circle, Suite 630, Dept. RC, Washington, DC

20036-1183.

Wholey, J., Hatry, H., & Newcomer, K. (Eds.) (2010). Handbook of practical program

evaluation (Third Edition). Jossey-Bass: San Francisco.

http://www.luc.edu/


Appendix A: Arrupe College Supporting Documents

Arrupe College Presentation


Statistics for Class of 2017

Class of 2017 First-Year Student Profile

159 students

Top Feeder Schools

1) Chicago Academy High School 10 enrolled students

2) Amundsen High School 9 enrolled students

Cristo Rey Jesuit High School

3) Chicago Bulls College Prep 8 enrolled students

Jan May July AugMarchFeb April June

TIMELINE

ACHIEVABLE. ACCESSIBLE. AFFORDABLE.

JANUARY 1

Fill out

FAFSA

MARCH 1

Application

deadline

MARCH 15

Decision Day

MAY 1

Reserve

your spot

SUMMER

Orientation

Classes begin

Mid August

NEXT STEPS


Our Lady of Tepeyac

4) Solorio Academy High School 7 enrolled students

5) Josephinum Academy 6 enrolled students

Senn High School

Demographics

Hispanic/Latino 113 (71%)

African-American 35 (22%)

White 7 (4%)

Asian 4 (3%)

Female 91 (57%)

Male 68 (43%)

Undocumented 34 (21%)

Public high schools 71 (45%)

Catholic high schools 44 (28%)

Charter high schools 44 (28%)

Average ACT 18 ACT Range 12 – 28 ACT Middle 50%

Range 16-20

Average GPA 2.8 GPA Range 1.74 – 4.42


Appendix B: STARS Program Supporting Documents


Appendix C: Logic Model for Mentoring


Appendix D: Survey


Appendix E: Survey Construct Map

Logic Model Component

Q# Question

Answer Type Answer Options

General First Term Involvement

1.1

I feel confident in my academic ability to complete Arrupe's associate

degree program.

Likert Scale (grid, using radio buttons)

Strongly Agree, Agree, I do not Agree or Disagree, Disagree, Strongly

Disagree

1.2

I am satisfied with my current GPA. Likert Scale (grid, using

radio buttons)


Disagree

1.3

I feel the academic resources at Arrupe College have contributed to

my overall academic success.



Disagree

1.4

I feel a sense of belonging to the Arrupe community. Likert Scale (grid, using

radio buttons)


Disagree

1.5

I have made friends at Arrupe. Likert Scale (grid, using

radio buttons)


Disagree

1.6

I interact with my classmates outside of the classroom. Likert Scale (grid, using

radio buttons)


Disagree

1.7

I interact with my professors outside of the classroom. Likert Scale (grid, using

radio buttons)


Disagree

1.8

I attend events with my classmates at Arrupe and/or Loyola. Likert Scale (grid, using

radio buttons)


Disagree

1.9

I attend events with my classmates outside school. Likert Scale (grid, using

radio buttons)


Disagree

Demographics

10

How do you identify your racial background?

Multiple Choice/Open-ended option

Black, American Indian/Alaskan Native, Asian American, Latino/Hispanic, Middle Eastern/Northern African, Native Hawaiian/Pacific Islander,

White, Multiracial, Racial Identity Not Listed, I prefer not to respond, Other

11

How do you identify in terms of gender? Multiple Choice/Open-

ended option

Female, Male, Transgender, Gender Queer/Non-Conforming, Gender Identity Not Listed, Prefer not to

respond, Other

12 What is your Grade Point Average

after the first term? Multiple Choice 4.0-3.5, 3.49-3.0, 2.99-2.5, 2.49-2.0,

Below a 1.99


S-T Outcome #1: Students are able to talk about what type of students they are and how identities may affect them.

8

If you were to participate in a mentorship program, would you want

your partner to share similar identities to you? Multiple Choice Yes, No

S-T Outcome #2: Students develop a greater sense of belonging and the college has higher retention.

2.1

Interacting with faculty outside of the classroom.


Very Interested, Somewhat Interested, A Little Interested, Not At All

Interested

S-T Outcome #3: Student develops a peer and network of support at the college.

2.2

Interacting with students outside of the classroom.



Interested

S-T Outcome #4: Students begin to succeed at a higher level in and outside of the classroom.

2.3

Meeting with professionals from my field of study.



Interested

S-T Outcome #5: Increase loyalty to college.

2.5

Attending Loyola events. Likert Scale (grid, using

radio buttons)


Interested

M-T Outcome #1: Students academic performance increases.

5

Do you feel that a mentorship program would benefit your

academics? Multiple Choice Yes, No

M-T Outcome #2: Students develop a peer group/network at the college.

2.4

Attending social events with my classmates at Arrupe.



Interested

M-T Outcome #4: Students are more comfortable and connected to community.

2.8

Attending events in the Chicagoland area. Likert Scale (grid, using

radio buttons)


Interested

3 Would you like to feel more a part of

the Arrupe community? Multiple Choice Yes, No

6

Which of the following mentors would you find most helpful? Multiple Choice/Open-

ended Option

Faculty member, Staff member, Loyola student, Arrupe student, Professional

working in my desired field, Other


M-T Outcome #5: Students continue with studies or academic opportunities.

2.7

Participating in a mentoring program.



Interested

7

How often would you like to receive academic mentorship? Multiple Choice/Open-

ended Option

Once a week, Once a month, Twice a month, Once a semester, Only when I

need help, Other

M-T Outcome #6: Maintain a 2.5 GPA or higher.

2.6

Attending workshops to better my academics. Likert Scale (grid, using

radio buttons)


Interested

L-T Outcome #1: Increase loyalty to college.

4

Do you believe a mentorship program could help make you feel more a part

of the Arrupe community? Multiple Choice Yes, No

L-T Outcome #2: Students efficacy and capacity for leadership has increased.

9

Which would you rather be?

Multiple Choice Mentee, Mentor


Appendix F: Focus Group Protocol

Arrupe College Focus Group Protocol: Mentorship

Introduction of Focus Group: On behalf of Arrupe College, we are so grateful that you are

taking time out of your days to share your experiences of being an Arrupe College student. My

name is (Evaluator A) and I am one of the evaluators of the program. I am also joined by

(Evaluator B). Today’s goal is to hear from you about your experiences at Arrupe. By talking

with us we hope to provide feedback and more in depth responses in addition to the surveys

given previously. We hope this information will help the Arrupe staff develop additional support

for the students. Please note the information you share today with use will not be connected to

individual names or identities.

Housekeeping: This interview will be 90 minutes and will be recorded by (Evaluator B) in order

for us to be more present in the conversation. The evaluators will be the only ones who will hear

the audio recording. We will use guided questions for this conversation, but want this to be as

honest and authentic as possible. Please share stories, build off of each other’s stories and views,

and dialogue with each other.

Prior to starting the conversation, we would like to give you the opportunity to review the

consent form for today’s session. There are copies of the consent form sitting in front of you.

The highlights of the consent form are:

The session today will last 90 minutes.

All information will be confidential.

We ask that you keep each other’s information and things you hear today as confidential.

No names are shared from this focus group.

This focus group is voluntary and you may leave at any time.

Are there any questions at this time regarding the consent form?

Please take some time to review the consent form, and if you are comfortable you may sign and

date it. You can give your signed consent forms to (Evaluator B). Then please help yourself to

Chipotle and drinks and we will begin once everyone is sitting down.

Introduction of Participants: To start off, we would like everyone to introduce yourselves.

Please state your name, what high school you went to, and why you came to Arrupe College.

Transition: Thank you for sharing and introducing yourselves. We would now like to begin this

conversation by talking about your general thoughts and experiences at Arrupe College.

General/Overall Thoughts on Arrupe College:

Main Question: What has your experience at Arrupe College been like thus far?

Can you please describe what your experience academically has been like at Arrupe?

Can you please share what your social interactions have been like at Arrupe?

Can you please describe what type of support you have received at Arrupe, if any?

Transition: Thank you for sharing. We would now like to ask you more about the potential need

for mentorship at Arrupe College.

Mentorship:


Main Question: How many in the group, if any, would be interested in a mentoring

program at Arrupe?

Please describe how peer mentorship would benefit Arrupe College students.

If not interested, why not?

For those interested, what would you hope to get out of a mentoring program?

With whom would you like to be mentored by?

o Other Arrupe students, Loyola University students, faculty, staff, Alumni?

How often would you like to receive mentoring?

In what capacity, academics, social, career, ect?

Transition: Thank you for sharing your thoughts. We will now move on to other

recommendations and changes you see important to your experience at Arrupe.

Final Thoughts:

In what ways, if any, can Arrupe improve your overall experience at the College?

What if anything should be retained but improved to improve your experience?

Conclusion: We have reached the conclusion of our conversation. Do you have any final

thoughts, questions, comments, or feedback that you would like share?

We want to thank you for your participation and conversation today. We understand that you are

very busy and your time today will be very helpful in evaluating a mentorship program at Arrupe

College. Please feel free to contact us with any questions or further comments may arise after we

leave this space.


Appendix G: Consent Forms

Focus Group Consent Form

You are about to take part in a focus group to help Arrupe College gain better

understanding of how a mentorship program would benefit students. You are being asked to

participate in this focus group based on your experience at Arrupe and the need for a mentorship

program.

This focus group will be used for Arrupe staff to assess the importance of a mentoring

program. The information that is shared in this focus group will only be shared with Arrupe staff

and all identifying information of participants will be confidential. This session will be audio

recorded, however, the recording will be destroyed after it has been transcribed, coded, and

analyzed. Your participation in this focus group is voluntary.

Purpose:

The purpose of this focus group is to gain information about Arrupe College student experiences

and needs in regards to a mentorship program.

Procedures:

This session will last 90 minutes. Once you agree to be a participant, you will be asked a series

of questions pertaining to your overall experience and need for a mentorship program. There are

no right or wrong answers and we will ask that you respond honestly and openly. Please know

that all questions are optional and you do not have to answer any questions that you choose not

to respond.

Risks and Benefits:

There are no foreseeable risks involved in participating in this focus group beyond that

experience in everyday life. Benefits to participating in this focus group may include further

involvement with a mentorship program at Arrupe and deeper understanding of learned

outcomes experienced as a result of being a student of Arrupe College.

Confidentiality:

Any information that is obtained during the focus group will remain confidential. In order to

maintain confidentiality:

Recordings of the interview will be destroyed after they are transcribed, coded, and

analyzed

Your real names will never be used in any reporting of data and during analysis

Only evaluators will have access to the taping of focus group, which will be kept on a

password-protected computer.

All participants in the group will be asked to keep what is talked about private, but this

cannot be assured.

Voluntary Participation:

All participation in this focus group is voluntary. During any point in the focus group, you are

free to withdraw from participation. Please know that you are not required to stay for the entire

time.


Contacts and Questions:

If you have any questions or concerns about the focus group, you may contact Emily Ferron at

[email protected].

By signing below, I confirm that I have read the above statements, agree to the terms of the focus

group, and wish to participate.

Signature:_____________________________________________________________________

Date:_________________________________________________________________________

mailto:[email protected]


Appendix H: Email Invitations

Greetings (Student),

I hope this email finds you well! As a student at Arrupe College, you are invited to participate in

a focus group to provide insights on a mentorship program.

The purpose of this focus group is to understand the impact of a mentorship program at Arrupe

College. The focus group will be held on (date) at (time) and will last for 90 minutes.

Your participation is voluntary, but we hope that you will aid in our evaluation of a mentorship

program at Arrupe College. Please note that your conversation will remain confidential.

Please direct any questions to Emily Ferron at [email protected].

Thank you,

The Arrupe College Team

mailto:[email protected]


Appendix I: Timeline

Activity 12-Dec 11-Jan 25-Jan 29-Jan 29-Feb 14-

Mar 18-

Mar 18-Apr 16-

May

Review Evaluation Plan with Dean of Students

X

Review Surveys & Focus Group Protocol with Dean of Students

X

Pilot Test Survey with Arrupe Student Government

X

Revise Survey based on feedback

X

Select participants for focus group

X

Announce survey through email

X

Email invitations for focus group

X

Conduct survey at Town Hall Meeting

X

Email survey to absent students

X

Conduct focus groups

X

Final email reminder about survey for absent students

X

Survey Closes X

Quantitative Data Analysis

X

Qualitative Data Analysis

X

Final analysis of integrated data

X

Recommendations presented mentorship program

X


Appendix J: Budget

Item Item Type Quantity Cost

Transcription - 2 focus group transcripts (work study student) Personnel 1 $0

Focus Group Facilitator - Intern Personnel 1 $0

Campus Labs Analytic Tools N/A $0

Emails Materials N/A $0

Arrupe Store Gift Card Incentives 1 $50

Data Storage (on campus computer & drive) Materials N/A $0

Audio Recorder (rent from Digital Media Lab) Materials N/A $0

Room for Town Hall Survey (Arrupe Computer Lab) Materials N/A $0

Rooms for Focus Groups (Arrupe Classrooms) Materials N/A $0

Printing for reports (included in departmental budget) Materials N/A $0

Chipotle for each focus group ($90/session) Incentives 2 $180

Beverages ($5/session) Incentives 2 $10

Total Cost of Assessment $240


Appendix K: Power Point Presentation