Aivazian-Preview Days Evaluation Project

Running head: GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 1

Graduate Assistant Preview Days Evaluation Plan

Russell C Aivazian & Diego Donna

Loyola University Chicago

GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 2

Table of Contents Introduction 3 History of Preview Days 4 Inputs 6 Purpose and Assumptions 7 Format and Structure 10 The Buddy Program 10 Higher Education Program Information 11 Workshops and Panels 11 Campus Tours 13 Socials 13 External Factors 14 Stakeholders 15 Evaluation Approach 16 Quantitative Approach 18 Population and Sampling 18 Design and Instrument 18 Pilot Testing and Implementation 21 Analysis and Results Presentation 23 Qualitative Approach 24 Participants 25 Interview Protocol 26 Implementation 27 Limitations 29 Budget 30 Timeline 31 Next Steps 32 References 33 Appendices

A. Logic Model 34 B. Previous PD Schedules 35 C. Preview Days 2010 Evaluation Findings 38 D. 2013 Instrument 46 E. Survey Instrument 58 F. Construct Map 67 G. Email Templates 73 H. Interview Protocol 75 I. Interview Consent Form 78 J. Interviewers Notes 79 K. Email Templates 80 L. Initial Codes 81 M. New GA Compensation and Structure Email 82 N. Timeline 85 O. Budget 86 P. Presentation 87


Specifically for students looking to pursue a degree in student affairs, interview and

outreach programs are one of the ways students look to better understand campus culture and

make a more informed decision on the school where they want to pursue their masters degree.

Depending on the program, these programs are often used as requirements for admission or

consideration of an assistantship. At Loyola University Chicago (LUC), the Preview Days

program is a partnership between the School of Education, Academic Affairs, and the Division

of Student Development (DSD) to help recruit and fill Graduate Assistant (GA) positions across

various functional areas within the university. The GA positions filled during Preview Days

supplement the coursework in various masters programs across the university, including the

Master of Education in Higher Education program (Division of Student Development, n.d.).

Each assistantship allows students the ability to engage in professional development

opportunities, while receiving financial support for their graduate studies (in the form of tuition

remission and a stipend, depending on position). Graduate Assistants work in departments

across the University, including, but not limited to, Student Activities and Greek Affairs, Student

Diversity and Multicultural Affairs, Residence Life, Academic Affairs, and Campus Ministry.

GAs are typically offered ten month contracts, lasting the duration of their academic program

(two years for Higher Education masters students) and work about twenty hours per week, on

average (Division of Student Development, n.d.).

In addition to providing a space to conduct interviews for the various GA position

openings, Preview Days provides an opportunity for the Universitys partners to interact with

potential graduate students and help candidates articulate the Loyola GA experience.

Throughout the two days of the program, participants are invited to attend various lectures,

panels, tours, and socials in order to better understand the institution and the opportunities


available at LUC. Even though the Preview Days program generally caters to Higher Education

masters students, participants from other graduate preparation programs (Social Work, Pastoral

Studies, Business, etc.) are invited to attend as well.

History of the Preview Days Program

Prior to 2006, each department within the Division of Student Development (DSD) and

Academic Affairs used their own process to recruit, interview, and hire Graduate Assistants

(GAs) for their open positions. Due to the lack of consistency among the various departments,

GA candidates were met with a confusing process that required them to submit similar and

redundant application materials for each GA position. Jack McLean, JD, Assistant Vice

President for the DSD, notes that even though many departments shared application materials

and processes, there was little collaboration to create a cohesive process and hiring timeline

(personal communication, September 25, 2014). Driven by requests from individual departments,

the DSD was charged with creating a Preview Days experience that introduced candidates to

Loyola and reflected the changing demographic of the Higher Education masters program

(specifically, increased student interest from outside the Midwest). Under McLeans leadership,

the GA Recruitment and Selection Team was created to help streamline the entire process, which

included planning the components of the Preview Days program.

The original Preview Days program was first introduced in 2006 as the Graduate Intro

Days, which invited GA candidates to Chicago to interview for all open positions in a more

centralized process. As graduate enrollment increased in the Higher Education program and the

DSD expanded their GA program, the Division focused on creating a development plan for their

GAs. This included the creation of the GA Formation Council and the introduction of the GA

orientation, which was used as a supplement to the Intro Days program in 2008. In 2009,


under new leadership, the DSD revised the GA recruitment and selection process to increase

collaboration between the Higher Education program and the various divisional partners in order

to create a more meaningful experience for prospective Graduate Assistant (GA) candidates. It

was at this point that the Division of Student Development (DSD) changed the name to Preview

Days, signaling a renewed focus on introducing GA candidates to the University and the GA

experience. McLean explains that the initial vision for Preview Days was modeled after the

program at Seattle University. In this new model, interviews would be paired with sessions

about the Loyola GA experience and information about moving to and living in Chicago (Jack

McLean, JD, personal communication, September 25, 2014).

Since 2009, the GA Recruitment and Selection Team have worked to create a more

coordinated and centralized process by evaluating and assessing the various components of the

process (see Appendix C and D for a summary of the evaluation findings from 2010 and the

evaluation instrument used in 2013). One of the more significant changes is the move from

paper application materials to a fully electronic system of gathering and evaluating applicant

materials and demographic data (J. Curtis Main, personal communication, September 25, 2014).

Using this new online system, GA candidates are able to submit their applications in one

centralized location and indicate their intention to attend the Preview Days program. On the

administrative end, hiring supervisors (individuals selected by their departments to coordinate

the hiring process for their GAs) are able to gather candidate information and easily identify

possible candidates for their open positions. This also allows for more consistent communication

from the DSD and an easier process for GA candidates to inquire about the assistantship

offerings and Preview Days program. As a result of the Divisions efforts to create a more

streamlined process and training sessions offered by the Higher Education program to help GA


supervisors understand the graduate application process, the DSD has seen an increase in the

number of submitted applications and overall attendance at Preview Days.

Inputs to Preview Days Program

The Preview Days program requires collaboration between DSDs departments and

partners in order to create a positive experience for prospective GAs. As our logic model

(Appendix A) illustrates, the execution of the Preview Days program requires the interaction and

use of four groups of inputs: prospective Graduate Assistants (GAs), time, resources, and

personnel. Since the Preview Days program is planned for the benefit of prospective GAs, it is

important to consider their input and behaviors when considering the many components of the

program. Without the attendance of prospective GA candidates, the Preview Days program

would not achieved its desired goals and outcomes (discussed in the next section).

Jack McLean and J. Curtis Main (personal communication, September 25, 2014) both

expressed the considerable amount of time that goes into the planning and execution of the GA

recruitment and selection process. Beginning in late June, members of the Division of Student

Development (DSD) and the Higher Education program get together to evaluate the process from

the previous year and set internal and external deadlines for the departments and prospective GA

candidates. During this planning stage, the various departments work to define the dates for

Preview Days (usually late February) and the timeline for the planning and execution of the

program. In addition, individual departments set aside time to review applications and plan for

the potential openings in their departments for the next academic year.

The Preview Days program must also take into account the various resources needed to

provide an impactful experience for the potential GA candidates. During the program,

participants are also treated to a selection of food from various Chicago restaurants, which


require the commitment of monetary resources from the Division of Student Development

(DSD). In addition to the cost of food, the DSD must cover the cost of printing all materials

(Preview Days guide, nametags, and handouts) used throughout the program (J. Curtis Main,

personal communication, September 25, 2014).

Finally, the Preview Days program enlists the participation of the personnel within the

DSD in various capacities. In order to plan and execute the entire GA recruitment and selection

process, the DSD relies on two committees: the Graduate Assistant Recruitment and Selection

Team (GARST) and the Preview Days Planning Committee (PDPC). Whereas the GARST

works to plan the entire recruitment and selection process for the DSD, the PDPC works to plan

the various components of the Preview Days program. Both of these committees require the

DSDs personnel to commit time to carrying out the process and work in concert to create a

positive experience for prospective GAs. In addition to serving on one of the planning

committees, Divisional staff and current GAs are also invited in a number of capacities

(presenting, attending a social, leading a campus tour, etc.) to interact with the prospective GAs

throughout Preview Days. Since the Preview Days program is planned in partnership with the

Higher Education program, faculty members are also invited to interact with candidates and

provide an introduction to the Higher Education program (Jack McLean, JD, personal

communication, September 25, 2014). It is assumed that throughout this process, University and

Divisional personnel will be available and willing to dedicate their time and resources to the

execution of this program (Appendix A).

Purpose and Assumptions of the Preview Days Program

Even though the Preview Days program has existed in its current form since 2009, the

program lacks explicit learning outcomes to help guide the planning and execution of the


program (Jack McLean, JD, personal communication, September 25, 2014). When assessing a

specific program, stated learning outcomes are essential to understand the reasons why the

program exists and what the program aims to achieve. As Weiss (1998) explains, when goals

are unclear or ambiguous, more than the evaluation can be affected. Where there is little

consensus on what a program is trying to do, the staff may be working at cross purposes (p. 53).

In the absence of specific learning outcomes for the program, McLean explains that the primary

focus of the Preview Days program is to expose participants to Loyola and the Division of

Student Development (DSD) in order to assess whether or not Loyola will be a good fit for a

candidates graduate studies. The Preview Days program works to achieve this outcome by

partnering with the Higher Education program to expose students to the expectations of the

program, introducing the Jesuit mission of Loyola University Chicago, helping departments

coordinate their outreach to potential candidates, and expose candidates to the diverse pool of

applicants and staff within the DSD. By highlighting these specific components of the Loyola

GA experience, McLean hopes that prospective GA candidates will walk away from Preview

Days having a better understanding of their fit at Loyola as well as the unique aspects of the

DSDs vision for the GA experience.

When assessing the level at which these learning outcomes occur, it is important to

highlight the assumptions that can be made about the DSDs approach to the Preview Days

program (listed in Appendix A). As we will discuss later, there is no formal requirement that

each participant attend all of the programs offered during Preview Days. Candidates may either

be taking part in interviews or choose to not attend various programs. By structuring the

Preview Days program in this way, the Division of Student Development (DSD) assumes that

participants will attend all applicable workshops and sessions during their time at Preview Days.


Various factors may lead to a participants decision to attend a specific session during their

experience, which may change their perspective on the usefulness or impact of the Preview Days

program on their choice to attend Loyola for graduate school. This assumes that participants will

be aware of all the opportunities at Preview Days and will know what sessions to attend in order

to better understand the Loyola Graduate Assistant (GA) experience.

In addition, the program assumes that potential GA candidates will have little to no prior

knowledge about Loyola University Chicago, Loyolas GA experience, and the city of Chicago.

In our examination of the Preview Days schedule and program offerings, discussed later

(Appendix B), many of the sessions are planned and presented to candidates with the intent to

educate candidates about the unique opportunities at Loyola and in Chicago. For example, there

is time set aside in the schedule to talk about the mission and vision of Jesuit education (Jesuit

Education 101: Jesu-WHAT?). When planning this program, the assumption is that the

participants posses little understanding about the Jesuit educational practices. Even though many

of the participants may not have knowledge of the Jesuit education model, those that have prior

knowledge about the topic may not experience the same impact as a result of the session. Using

this framework to guide the participants experiences during Preview Days, there is an

assumption that the programs presented will meet the needs and retain the interest of prospective

students. In its current format, students are not asked prior to Preview Days about the questions

they may have regarding Loyola and the Loyola GA experience. Previous evaluations have been

conducted, focusing on the logistics (communication, flow of the schedule, etc.) and the

programmatic aspects (various sessions) of Preview Days (Appendix D). Absent in previous

evaluations is a focus on the participants overall experience (in relation to the learning

outcomes) as well as an assessment of the information participants wish they would have


received during the process. The Division of Student Development (DSD) does use previous

evaluations (see Appendix C) to frame their conversations about individual sessions, however

the evaluations are used to understand how the content should be presented during Preview Days.

Format and Structure of Preview Days

The Preview Days program, in its current form, takes place over two days and is held at

Loyolas Water Tower and Lake Shore Campuses in late February (see Appendix B for previous

Preview Days schedules). Candidates are invited to attend if they are offered an interview by the

hiring supervisors throughout the Division of Student Development (DSD) and are scheduled

according to the number of interviews offered to the candidate. Typically, the Preview Days

program begins with the Higher Education program introduction and dinner on the first day of

the program and ends once the candidates interviews have completed on the second day.

During the second day of the program, interviews are conducted simultaneously with the

programs planned by the Preview Days Planning Committee (PDPC). Over the past three years,

the opportunities available to Graduate Assistant (GA) candidates consist of the buddy

program, Higher Education program information, workshops and panels, campus tours, and

socials, which are spread throughout the two days of Preview Days (see Appendix A).

The Buddy Program

Unlike other institutional programs, candidates are not required to attend the Preview

Days program in order to be considered for a Graduate Assistantship or admittance into the

Higher Education program. For the last few years, PDPC has paired current GAs with applicants

as a way to bridge the gap between individuals who attend the program and those that choose not

to attend. Current GAs are paired with six to ten GA candidates and are instructed to reach out


via email and act as a resource if GA candidates have any questions about either the recruitment

and selection process or about Loyola.

Higher Education Program Information

During the first day of the program is the Higher Education Program Overview &

Dinner, which usually takes place at Loyolas Water Tower Campus (Appendix B). During this

session, candidates hear from program faculty about the various components of Loyolas

program and are encouraged to interact with faculty and current students over dinner. Since an

outcome of the Preview Days program is to be introduced to Chicago, the Preview Days

Planning Committee uses meal times as a way to expose candidates to the various food offerings

in Chicago.

In addition to the information session on the first night of the program, Higher Education

students are encouraged to attend an optional class during on the second day of the program. In

the past, students have typically attended the Student Development Theory class, but are asked to

RSVP in advance since there are limited seats in the classroom. This offers candidates another

opportunity to interact with the faculty and understand the academic experience at Loyola.

Since many of the Graduate Assistant (GA) positions do not require students to be

enrolled in the Higher Education program, candidates from other programs (Social Work,

Pastoral Studies, Business, etc.) are invited to interview during the Preview Days program.

These candidates are invited to the activities on the first night of the program, but are not

required to attend and encouraged to reach out to their respective programs of study.

Workshops and Panels

Throughout the Preview Days program, GA candidates are invited to attend various

sessions that help them to understand the Loyola GA experience. Over the years, the overall


framework of the sessions has remained unchanged (Appendix B), even though the content

presented may look different from year to year. Starting with the first night of the program,

candidates participate in a panel of current masters students in the Higher Education program,

who speak to their experience with the academic components of the program. The Preview Days

Planning Committee (PDPC) works to gather a diverse set of students who have varying

experiences with the Higher Education program (full and part time, current GAs, post-

undergraduate experiences, etc.) in order to offer different perspectives about the program. In

addition to the panel on the first night, an additional panel is offered during the second day,

focusing on the Graduate Assistant (GA) experience and the relationship between GAs and

University personnel. During both panel discussions, prospective GAs are encouraged to ask

questions that help to clarify their intent to join Loyola as a GA.

In addition to the panels offered during the Preview Days program, GA candidates are

also invited to attend sessions and workshops about the Jesuit and social justice mission of

Loyola as well as the opportunities available to GAs living in Chicago. Beginning with the

Division of Student Development welcome, usually delivered by the Vice President for Student

Development, candidates are exposed to the Loyola student GA experience. As an introduction

to the Jesuit identity of Loyola and the Jesuit education model, Jesuit Education 101: Jesu-

WHAT? is presented by staff members from Campus Ministry. Since many candidates come to

Preview Days with little knowledge about Jesuit education, this session serves as a way for

students to understand the characteristics of Jesuit education and understand how Loyolas Jesuit

identity may frame their work at the institution. In conjunction with faculty members in the

Higher Education program and divisional staff, candidates are also engaged in a discussion about

social justice and how it informs professional practice at Loyola. Given that the Higher


Education program has a focus on social justice, this session has historically allowed candidates

to engage with each other in conversations about social justice and introduces them to the types

of conversations and material covered within the classroom.

Finally, the Moving to Chicago presentation offers candidates an opportunity to learn

about living options in Chicago as well as the opportunities available in the city. Since most

Graduate Assistants (GAs) will live off campus, this session is intended to provide information

about living in the city and is usually presented by GAs and the Division of Student

Development (DSD) staff who live in the neighborhoods surrounding Loyola. All of these

programs are intended to help candidates assess their fit at Loyola as well as educate them

about the mission of Loyola.

Campus Tours

During the Preview Days program, participants have many opportunities to tour the Lake

Shore and Water Tower Campuses. During these tours, led by current GAs, candidates are

exposed to the physical layout of Loyolas campuses and are introduced to the various campus

resources graduate students use frequently. During the tour, candidates are able to see the

various facilities and become somewhat familiar with the campus. This is another opportunity

for participants to interact with current GAs and gain a better feel for the campus environment at

Loyola.

Socials

In addition to the Higher Education Program Overview and Dinner on the first night,

candidates are given the opportunity to interact with undergraduate and graduate students, DSD

staff, and faculty members. In previous evaluations, the Preview Days Planning Committee has

observed that candidates have requested more time set aside to meet with the members of the


Loyola community. Throughout the past few years, these social aspects have been included into

meal times, structured settings (Social with Grads and Undergrads), and throughout the

interview process. The Preview Days Planning Committee has worked to make sure candidates

can have their questions answered by staff members and current Graduate Assistants (GAs)

throughout out the process. DSD staff operate a Hospitality Booth throughout the second day

of the program and current GAs are able to volunteer to mingle with candidates in the interview

preparation space to calm nerves and serve as a resource.

External Factors

In addition to the assumptions made by the DSD about the achievement of the learning

outcomes, there may be various external factors, which may lead to obstacles and disruptions in

the execution of Preview Days (Appendix A). Since the program has seen an increase in the

number of candidates from outside the Chicago area, the cost of attendance has been a factor to

which Jack McLean and J. Curtis Main are sensitive (personal communication, September 25,

2014). Many candidates travel via different forms of transportation, which may be cost

prohibitive or highly susceptible to changes in weather. Since Preview Days normally takes

place in late February, candidate travel may also be impacted by the cold and unpredictable

weather in Chicago.

The extent that the Preview Days program impacts a candidate may also be affected by

their previous beliefs and assumptions about Loyolas program. For example, if a candidate

labels Loyola as more desirable than another institution (assuming their process includes

multiple institutions), they may respond more favorably to the experiences during Preview Days.

In contrast, candidates who may be on the fence may look at the Preview Days experience

more critically in order to assess their interest in the Higher Education program and GA


experience. In addition to these external factors, institutional factors may positively or

negatively impact a candidates experience. The programs outcomes may or may not be met

depending on the availability of University resources (physical, monetary, and/or human), for the

Preview Days program. The institutional climate (demographics, facilities, etc.) may not be

desirable for candidates, which could impact their view of the institution and their perceived fit.

Finally, a candidates choice to attend Preview Days or attend Loyola may be impacted by the

changes to assistantship positions and stipends. Within the next two years, the Division plans to

increase stipend amounts while decreasing the total number of assistantships across the Division,

which may impact the number of candidates who attend Preview Days (Appendix M).

Stakeholders

Before understanding our approach to the evaluation of Preview Days, it is important to

understand the various stakeholders involved in this program. Wholey, Hatry, and Newcomer

(2010) explain that stakeholders are individuals, groups, or organizations that can affect or are

affected by an evaluation process or its findings (p. 31). In our conversations with Jack

McLean, JD (personal communication, September, 25, 2014), it is clear that the primary

stakeholders for this program are the prospective GA candidates, Higher Education program

administrators, and divisional staff that plan and execute the Preview Days program.

Specifically for the Division of Student Development (DSD), our evaluation of the Preview Days

program directly influences the prospective GA candidates. As Loyola aims to be more

competitive with their processes and GA offerings, it is important for the (DSD) to evaluate the

program and implement changes that reflect candidates needs. If candidates needs are not met,

they will likely not attend Preview Days or apply for open GA positions. This relationship is

also important for the members of the Higher Education administration. Even though there is not


a formal requirement to hold a GA at Loyola in the graduate program, the Preview Days

experience is an important piece in a students decision to attend Loyola. This is not the only

factor, but must be taken into consideration as we begin to form and create an evaluation plan for

the program (Jack McLean, JD, personal communication, September 25, 2014).

Finally, the personnel in the Division of Student Development who use the process to

recruit GA candidates would also be impacted by an evaluation of the program. Much like the

Higher Education program faculty, the Preview Days experience is an important step in selecting

the most desirable candidates for the open Graduate Assistant (GA) positions. As we mentioned

earlier, the absence of this streamlined and cohesive process caused confusion for GA candidates,

which may have persuaded them to consider other institutions for their graduate studies. If

candidate needs are not taken into account or met, hiring supervisors may not have a reason to

recruit their GAs through this process (or hire a graduate student for the position at all). As we

begin to understand the design of our evaluation, it will be important to consider these

stakeholders throughout the entire process.

Evaluation Approach

Our evaluation will be formative in nature as we seek to provide information that could

be used to make improvements. A formative evaluation is defined as one whose primary

purpose is to supply information for the improvement of a program (Fitzpatrick, Sanders,

Worthen, p. 20, 2010). Based on our conversations with stakeholders, we will be using a

combination of process and outcomes approach to the evaluation of Preview Days. The reason a

process approach is helpful when evaluating Preview Days is due to the purpose of a process

study. Such studies may focus on whether the program is being delivered according to some


delineated plan or model or may be more open-ended, simply describing the nature of delivery

and the success and problems encountered (Fitzpatrick et al., 2010, p. 26).

Since we are looking into the effectiveness of Preview Days we must look at the various

panels, workshops, and events that occur throughout the program. Given that a process study is

based on the delivery of a program and also takes into account the success and difficulties of the

program, this approach would be advantageous to the needs of the Preview Days program. An

outcome approach is defined as one that takes into account the changes in an audience as a result

of a program (Fitzpatrick et al., 2010). As we have previously mentioned there are several

learning outcomes that aid in describing the purpose of Preview Days. Since the nature of our

evaluation is focused on the delivery and execution of Preview Days we use the learning

outcome that McLean expressed to help guide our evaluation. The main focus of Preview Days

according to McLean is to paint a picture for participants of the culture here at Loyola and within

the DSD. By providing participants with this, Preview Days hopes to help participants discern

whether or not Loyola is a good fit for their graduate studies. In evaluating Preview Days and

its execution, it is vital that we keep this learning outcome in the forefront as we progress

through the evaluation.

The combination of process and outcomes approaches is appropriate as we seek to

address the effectiveness of the Preview Days program and the value it offers to the Loyola GA

experience. Since the process aspect looks at the delivery of the program, we will be looking at

the influence of individual components in relation to the learning outcomes to expose

participants to Loyola and DSD in order to assess whether or not Loyola will be a good fit for

a candidates graduate studies. As the outcome approach examines changes in an audience of a

program, we will be surveying those individuals who attended Preview Days. This also


addresses the learning outcome of exposing participants to Loyola and DSD. The strength of this

dual approach is that the process aspect lends itself to looking at both the individual components

and their execution in relation to Preview Days as a whole. The outcomes aspect of this

approach illustrates the effectiveness of Preview Days and allows to us to see the affect such a

program has on its participants. This way if any changes need to be made we can find out what

exactly needs to be change and how to make the necessary adjustments in order to ensure a better

delivery.

Although both process and outcome approaches provide strengths to this evaluation, there

are also weakness to each aspect. Even though the process approach lends itself for a closer

examination of individual components of the Preview Days program, it does not take into

consideration outside influences and variables that potentially affect each piece. The challenges

of an outcome approach is the willingness participants to speak to their experiences and the

potential for varying answers from year to year that would leave have an effect on any changes

that would need to be made. Taking this into consideration, we will explain the various

components and approaches to our evaluation plan.

Quantitative Approach

Population and Sampling

Our intended population of study will be the candidates in attendance during the Preview

Days (PD) program. We selected this group because the focus of this evaluation entails

observing a change within a selected population as a result of attending a program. By surveying

the candidates, we can begin to evaluate the delivery and effectiveness of Preview Days. Since

Preview Days is designed and executed with the candidates needs in mind, it is also important to

note that they are one of the primary stakeholders. Their input and feedback will provide insight


into any changes that need to be made and other areas of improvement. Since Loyola aims to

become more competitive in the search and hiring process of Graduate Assistant (GA)

candidates, the information gained by this survey will prove beneficial.

As a result of surveying the candidates who attend Preview Days, we will conduct census

sampling. On any given year, there are about 50-75 candidates who attend Preview Days. The

reason we have selected census sampling is that we are choosing all the individuals who attend

PD. In this case, since those attending PD will have their email accounts already on file, it will

be easy to reach out to them and send out invitations. Another reason that census sampling is

best serves our evaluation is because the design and implementation of our evaluation focuses on

the experience of the individual participant throughout PD. One of the strengths of using census

sampling is that it allows for researchers to select participants who are able to speak to their

experience, in this case attending PD, and will also be able to answer the questions posed in the

survey surrounding their experience. We define our population as people who attend PD in order

to look at specific outcomes that arise as a result of their attendance. When using census

sampling, one weakness can be the limited ability to make generalizations and conclusions due

to the small scope of the population sample. Additionally, another limitation to our survey is the

assumption that all the participants that are in the intended survey sample will answer the survey

in a timely manner presents a weakness and potential challenge to our evaluation. Our

anticipated response rate is at least 50 percent, which reflects our realistic goal based on previous

response on the assessment for Graduate Assistant professional development sessions. Even

though this number is less than the optimal achievement 70 percent identified by Wholey et al.

(2010), it will still yield results that will allow us to make conclusions about the Preview Days

program.


Design and Instrument

We will be utilizing a descriptive cross-sectional design in our evaluation of the Preview

Days program. As Creswell (2009) notes this is a pre-experimental design where we study the

participants and view the effects the Preview Days program has on them afterwards. It is our

belief that cross-sectional best serves our needs as we seek to distribute the survey once all the

candidates have finished attending Preview Days. Cross-sectional studies measure a specific

point in time, in this case at the conclusion of PD, in order to accurately evaluate the delivery of

a program. As we collect our responses, one of the limitations to this research design is the

validity of responses entered. Our challenge in using a cross-sectional design is that we are not

controlling any previous knowledge, experiences, or expectations surrounding Loyola and

Preview Days, which would hinder the strength to which we are able to assess the impact PD had

on the participants. Our intention is to deliver the survey as quickly as possible with frequent

reminder emails (Appendix G) to allow our participants easier time recalling their experience.

We hope to engage our participants in a deeper reflection in their experience through interviews

in our qualitative analysis that shall be discussed later. Our quantitative approach reflects the

assumptions outlined in our logic model (Appendix A) that our participants enter PD with little

or limited surround Loyola and the GA experience. Our goal in administering this survey is to

evaluate the extent to which the outcomes, given the assumptions, are reached within the

delivery of the PD experience.

A majority of our survey focuses on the extent to which participants believe they were able to

achieve the outcomes of the PD experience. This is assessed through a set of statements that

asked participants to rate their level of agreement using a Likert scale (See Appendix E for

survey instrument). Our Likert scale allows our participants to choose between five choices


including: strongly agree, agree, neither agree or disagree, disagree, and strongly disagree. One

of the strengths of the implementation of a Likert scale is that it allows us to assess general

trends and distribution of the data collected. It also paints a general picture of the level of

agreement respondents had towards the statements we made. Our challenge in using a Likert

scale is that the difference between each choice in the response scale is dependent on the readers

understanding of the scale itself and statement made. Since we are assessing the experience of

PD, we are concerned with the distribution of the responses rather than the average response

given that a numerical value would not have much significance. Another concern of this

approach may be that participants may pick the extremes or the middle of the response scale in

interest of time. We hope to mitigate this concern with the use of a large response rate and

evaluation of the descriptive statistics the survey produces.

We are also utilizing open-ended questions to provide a unique space for participants to voice

their opinion and clarify their responses. Unlike the Likert scale, participants are able to

articulate their response in a more complex and unique way that allows researchers better

understand their experience. Our open-ended questions ask respondents to identify any lingering

questions, experiences that were valuable or could have been improved, and a space for

comments and suggestions. A limitation to using these types of questions is the complexity and

level of detail that different participants may employ. Variety, detail, and the unpredictability of

responses can also be a limitation as it may be a burden on researchers to analyze quickly.

Additionally, to many open-ended questions can lead lower responses rates and incomplete

answers as survey fatigue may set in (Schuh & Associates, 2009). The responses to these open-

ended questions will be compiled and given to stakeholders in order to make more informed

decisions surrounding any changes or improvements to the program.


Pilot Testing and Implementation

In the implementation of our current form of the survey, it is best to pilot test it in order to

ensure the success of our instrument. We plan on pilot testing our survey with current GAs who

attended Preview Days. Our goal is to gain feedback from at least ten people in order to ensure

that our questions are readable and that the participants do not have any confusion regarding both

the questions and responses. Another reason for pilot testing our survey, as noted by Creswell

and Associates (2009) is determining if we need to make any improvements in the questions we

ask and if there are areas where we should add or remove questions. In pilot testing our survey,

we will also be testing our delivery method, administering the survey through the web in an

email format. The current GAs who will be pilot testing the survey for us will be asked to take

down any notes or suggestions that they see fit and email those suggestions once they have

completed the survey. If we need some clarity on their comments and suggestions then we will

email them to follow-up.

The timing of the distribution of the survey as noted by Creswell (2009) involves multiple

waves and reminders. The survey will be administered by the Division of Student Development

(DSD) Graduate offices email and that office will serve as the main contact. All the emails will

be sent out using the DSD email of [email protected], and the first wave will be an invitation

email sent out the following Wednesday at the conclusion of Preview Days (Appendix G). The

tentative date of the first email will be the 25th of February, and the first reminder will be the

following Monday March 2nd. The subsequent reminders will be as follows: March 4, March 6,

and March 9, with the survey closing on March 11. These reminders are important as they help

encourage the participants to complete the survey. An added benefit of distributing the survey

through email is that it requires minimal effort in accessing the survey, as it will already be


linked to the participant. Two limitations to distributing the survey through email is the potential

for an incorrect email being used and user error in accessing the survey.

Analysis and Results Presentation

All the data we gather from the survey be organized and describe. The only statistical

analyses we will be using will be descriptive univariate statistics. These kinds of statistics are

those intended to summarize information on a single variable (Wholey et al., 2010, p.

455). As previously noted, many of the questions are tied to an outcome listed in our logic

model (Appendix F) and will provide interval data. For our demographic and general

information questions we will be analyzing the data using frequency of responses. For our Likert

questions we will be analyzing the data using means and medians to help determine the

frequency and distribution of responses. Along with using means and medians to help describe

the data, we will be using ANOVA and r-square statistics in order to measure the interaction

among variables and the significance of variables to their full experience. In addition to these

statistics we will also run a Chi-square test to determine if there is a statistically significant

difference between the Preview Days experience and their demographic characteristics. We

anticipate using SPSS as our statistical software to translate the data into these measures (which

is available on University computers).

We will be presenting these results in our final report using tables and figures that will

contain both the questions and the corresponding scores. There will also be an executive

summary to help the reader grab a big picture of the results and make it easier for them to

comprehend the results. In the executive summary there will be information containing our

purpose, several points that the reader should take notice of, and our own recommendations as


part of the evaluation. Our recommendation will be a result of our own interpretation of the data

collect and organized.

Qualitative Approach

In combination with the survey conducted at the conclusion of the Preview Days program,

we will conduct interviews with participants in order to better understand the achievement of our

outcomes (Appendix A). Utilizing a qualitative approach will allow us to better understand the

experience of candidates and explain phenomena that are presented during the administration of

our survey. Recognizing that our quantitative approach attempts to describe the experience of

Preview Days participants as a whole, our qualitative measure will help identify the nuances of

each students experience with the program. Our qualitative approach will use semi-structured

interviews in order to engage participants in a reflection about their experience and identify their

interaction with specific aspects of the Preview Days experience. Semi-structured interviews

combine directed interview questions with supplemental questions (known as probes) that ask

the participant to further explain their experience (Wholey et al., 2010). This approach allows

the interviewer to engage with the participant and clarify specific experiences that may be

essential to understanding an individuals experience with the Preview Days program.

We identified a semi-structured interview approach for the Preview Days program

because of the ability to conduct interviews easily while being able to have the freedom to ask

follow-up and probing questions to better understand their experience. In contrast to focus

groups, semi-structured interviews are conducted with one participant at a time (Wholey et al.,

2010). Even though focus group interviews allow participants to interact with each other and

speak about shared and individual experiences, semi-structured interviews will allow us to

quickly identify and interview participants without creating a focus group with participants


across the country. Using this one-on-one interview format, we will be able to easily schedule

interviews with participants and control for any technical difficulties that may arise from

conducting a focus group via telephone or videoconference.

Participants

The participants for the interviews will be selected from those who complete the survey

administered after the conclusion of Preview Days. While participants are completing the survey,

they will be asked if they would be willing to participate in an interview regarding their

experience. If they select yes, they are asked to provide an email address that they can be

contacted in order to schedule the interview. After the survey closes on March 11th, we will

begin to contact participants in order to schedule interviews for mid-April to mid-May. This

timeframe will allow us to analyze the data and edit our interview protocol, as appropriate. We

anticipate interviewing five Preview Days candidates and will compensate interview participants

with a $10 Amazon.com gift card.

Given that our interviews will last approximately 30 minutes, our sample size was

selected in order to conduct the interviews and analyze the data in order to make changes and

improvements to the next Preview Days program. The size of our sample for our qualitative

approach is a limitation to our assessment. Since we are not interviewing candidates that exhibit

a specific characteristic and relying solely on self-selection, we limit the generalizability of our

findings. However, we hope to use the interviews to not only understand the experience of

candidates during Preview Days, but to bring light to phenomena that may present itself during

the analysis of the quantitative data. Additionally, this evaluation component was chosen in

order to quickly gather information and recommendations and results in order for the Division to

consider any changes to the Preview Days program. If the Division of Student Development


wishes to better understand the experience of candidates who exhibit a specific characteristic

(demographic, programmatic, etc.), the qualitative approach would be the place to better

understand those variables. Our goal is to use the interviews in order to enhance the

interpretation of the quantitative data, while providing tangible examples to explain areas of

improvement for the program.

Participants will be selected randomly once the email addresses have been collected from

the survey instrument. Each email will be assigned a random number and will be placed in

ascending order. Once this has been completed, we will begin to contact participants in order of

their assigned number. When participants accept the invitation to be interviewed, we will work

to schedule their interview and send them the consent form (Appendix I), which outlines the

purpose of the interview. Additionally, the consent form will include demographic information

(similar to the information included in the survey instrument), which will be used during data

analysis after the interviews have been conducted. If participants decide to decline the offer for

the interview, we will select the next participant on the randomized list.

Interview Protocol

An interview protocol (Appendix H) will be used to conduct the 30-minute semi-

structured interviews. The overall structure of the interview will include an introduction, a set of

three foci with probing questions, and a conclusion. Prior to the interviews, we will conduct a

pilot test of our questions with current Graduate Assistants. Much like the quantitative approach,

pilot testing will allow us to make sure that the questions are constructed without any confusion

and will draw out directed responses. During the introduction of the interview, the interviewer

will explain the rationale of the interview and ensure that the participant has electronically signed

and returned their interview consent form (Appendix I). The interviewer will also reiterate that


the participants answers will be kept confidential and will not be traced back to their name or

candidate file. Since we are conducting these interviews during the timeframe when

assistantship offers are typically made, it is important that we ensure that the participants can be

honest with their responses, without any effect on their assistantship offer status. This timeframe

does present a limitation, but our goal is to ensure that the information provided during

interviews will not be traced back to their candidate file.

After the interviewer has explained the interview and asked for the participants verbal

agreement for continuing the interview and audio recording, the interviewer will ask the

questions outlined in the protocol. Given that we only have 30 minutes to complete the

interview, we have focused our time for the participant to answer three foci (all open-ended) with

additional probes provided for the interviewer. The interviewer will not be obligated to ask all of

the probing questions, but will be provided a list of probing questions in order to ensure that

participants are providing the desired response. Including the probing questions also allows

participants to provide more focused responses that will aid in our ability to analyze and interpret

the data. Our foci are intended to speak to our outcomes outlined in our logic model (Appendix

A), with specific emphasis on the medium and long-term outcomes. After the interviewer has

asked the three focused questions, they will be instructed to thank the participant and offer to

answer any questions about the interview. Additionally, participants will be asked to confirm

their email so that they can receive their compensation.

Implementation and Analysis

A current graduate assistant in the Division of Student Development (DSD) who is

recruited by the Preview Days Planning Committee will facilitate the interviews for this portion

of the assessment plan. Utilizing a current graduate assistant helps ensure the validity of the


responses because they are minimally involved in the hiring process for Graduate Assistants and

also able to relate to participants experiences during Preview Days. In partnership with the DSD,

the selected Graduate Assistant will work to schedule the interviews. All interviews will be

conducted via telephone and the interviewer will be asked to take notes regarding any major

themes or ideas that emerge during the interview (Appendix J). All notes will be collected in

order to be analyzed by the researchers, but will not weigh heavily in the final interpretation of

the data.

All interviews will be audio recorded, saved electronically, and transcribed by a graduate

student who will receive compensation (see budget in Appendix O). Once the interviews have

been transcribed, we will then engage in the process of descriptive coding in order to analyze the

data. Descriptive coding allows researchers to create categories as they examine transcripts,

while using initial codes developed in relation to the outcomes of the program (Wholey et al.,

2010). The initial codes we have developed (Appendix L) relate specifically to the learning

outcomes outlined in the logic model (Appendix A) and will be amended if specific phenomena

appears from the quantitative survey that becomes of interest to the DSD. Utilizing descriptive

coding for the interviews is helpful since this is the first time the DSD will take on a qualitative

approach, which involves coding. Descriptive coding allows the researchers to be flexible with

their coding structures and focus more on the specifics described in the interview. This type of

coding will be time intensive for the researchers doing the coding and those interpreting the

results (Wholey et al., 2010), but will help the DSD create a coding structure for future

interviews. We will also compare the candidates information with their choice to accept of

decline an assistantship using back-end coding once we have compiled all of the transcripts and

information.


After the transcripts have been coded, we will comb through the data using pawing in

order to observe the themes in the data as they emerge. As we examine the development and

interaction of the codes and the data, we will be able to look for specific themes and re-analyze

the data to identify other areas where those themes may emerge. Once we have examined the

data from the quantitative and qualitative measures, we will ensure validity through a process of

triangulation. As Wholey et al. (2010) explain triangulation helps to limit the bias that results

from relying on one source of information to make conclusions. Because we are asking

questions in both instruments about the outcomes and goals of the Preview Days program, we

can match the results of the evaluation approaches to identify any inconsistencies. If the data

does result in any inconsistencies, we will flag them and find ways to evaluate those phenomena

in future evaluations (Wholey et al., 2010). In addition to triangulation, we will also perform

member checking with the participants of our interviews. Member checking involves taking

initial interpretations findings, and descriptions back to the participantsto determine whether

they are accurate from the participants point of view (Schuh & Associates, 2009, p. 169). This

will ensure that our observations match the experience of the participants. Once the interviews

have been examined and validated, the conclusions will be paired with the quantitative data,

using direct quotes in order to support the conclusions within the final report.

Limitations

This evaluation plan will allow the Division of Student Development and the Preview

Days Planning Committee valuable information in order to make changes to the Preview Days

program to meet the needs of the Division and assistantship candidates. Even though the intent

of this evaluation is to aid in the execution of the Preview Days program, we must acknowledge

that there are a few limitations to our approach. Since we will be conducting the survey using an


online platform, our response level (50 percent) will be lower than what is optimal for

quantitative instruments (70 percent; Wholey et al., 2010). This will limit the generalizability of

our findings; however, we believe the 50 percent response rate will still let us make general

conclusion about the experiences that candidates have during Preview Days. We will hope to

increase our response rate through announcements during Preview Days and multiple reminder

emails after the program.

In addition to the response rate on the survey instrument, our small sample size for our

interviews presents further limitations. We only anticipate interviewing five participants, which

is about less than ten percent of the total participants in the Preview Days program. This size

was selected given that the DSD does not have any current qualitative evaluation structure for

the Preview Days program. Additionally, we are looking to gather results and make

recommendations before planning for the next Preview Days program. Selecting a small sample

size will be realistic and will provide quotes and observations that can be added to the

quantitative results.

Budget

As outlined in Appendix O, our total anticipated cost for this evaluation plan is $350.

The DSD already has access to CampusLabs and printing services, which will help deliver the

survey and print any paper materials and final report. The bulk of our budget will be spent on

the surveys in the qualitative plan. Participants will be compensated for their participation in the

interview with a $10 Amazon.com gift card. We anticipate the total cost of gift cards to be $50

and they will be distributed via email. Since our qualitative plan requires transcription and

coding, we will recruit a current graduate student to complete this step of our plan. Our

estimates indicate that it will take approximately 10 hours for the students to transcribe the two


and a half hours of audio recording (audio recorder provided by digital media services).

Additionally, we estimate coding to take a similar amount of time. We will pay our graduate

students $15 per hour, which we believe to be an acceptable incentive to complete the

transcription and coding. The Division may be able to explore options for students to perform

this work as part of an internship or summer project, if they wish to be cost conscious about the

evaluation program.

Timeline

Throughout the evaluation plan, a timeline has been outlined in order to provide further

direction for the evaluation of the Preview Days program. Our specific timeline is further

explained in Appendix N. After the completion of our evaluation plan, we will discuss the

findings with Jack McLean and J. Curtis Main to discuss our findings and understand the

feasibility of the study. We anticipate meeting with Jack and Curtis in December in order to

understand how to implement the evaluation plan within the Division. Once we have met with

Jack and Curtis, we will begin pilot testing our survey and interview protocol in order to

understand if any further changes need to be made to our plan. Based on this feedback, we

anticipate completing our survey instrument at the beginning of February, in anticipation of

Preview Days, which takes place later in the month. Once Preview Days is completed,

participants will be emailed the survey (with subsequent reminders) and be instructed to

complete the survey before March 11.

Once the survey is completed, we will email those participants who indicated that they

were open to being interviewed (after random selection is completed). We will schedule the

interviews and hold the interviews during April and May. The data will be transcribed and

coded after the completion of the interviews. Initial conclusions will be made based on the


evaluation results and sent to the participants who were interviewed for member checking. Once

we have received information back from the participants, we will compile the information into

the final report and inform the DSD of our findings and suggestions for changes to future

Preview Days programs.

Next Steps

As was mentioned earlier, we will meet with Jack and Curtis to discuss our findings and

understand the feasibility of our evaluation plan. We will send Jack and Curtis our plan

electronically and anticipate scheduling a meeting face-to-face with them to understand their

general reactions to the plan and any considerations we should take moving forward. Our hope

would be to assist the Division with this evaluation (in any capacity) and extend our help for the

planning and implementation of Preview Days 2015.

If the Division wishes to continue with this evaluation plan, it should consider finding

ways to perform an evaluation of students who are offered interviews but who choose not to

attend Preview Days. This information will allow the Division to assess any barriers to

attendance and better tailor the program to be available to a wider audience. Since the scope of

this evaluation plan is the execution of the existing program, we did not address this area in our

plan. Finally, the Division should also work with Preview Days stakeholders in order to better

target the quantitative and qualitative questions and understand how each office perceives the

Preview Days experience.


References

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods

approaches. Thousand Oaks, California: SAGE Publications.

Division of Student Development (n.d.). Graduate assistantships. Retrieved from:

http://www.luc.edu/studentdevelopment/assistantships/

Fitzpatrick, J. L., Sanders, J. R., Worthen, B. R., & Worthen, B. R. (2011). Program evaluation:

Alternative approaches and practical guidelines. Upper Saddle River, NJ: Pearson

Education.

Schuh, J. H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:

Jossey-Bass.

Weiss, C. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper

Saddle River, N.J.: Prentice Hall.

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.) (2010). Handbook of practical program

evaluation (Third Edition). San Francisco: Jossey-Bass.

GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN 34 Appendix A: Logic Model

Program: Graduate Assistant Preview Days Logic Model

Inputs Outputs Outcomes -- Impact Activities Participation Short Medium Long Prospective Gradate Assistants Time Resources: -Housing -Food -Preview Days guide -Facilities -Food Personnel: -Current Graduate Assistants (GAs) -Committees -Divisional staff -Hiring supervisors -Higher Education program faculty

Buddy System Interviews Programs: -Socials -Campus tours -Jesu-What? -Social justice introduction -Divisional overview -Classroom Experience -Workshops and panels

Prospective GAs Current GAs Divisional staff Higher Education program faculty Hiring supervisors

Understand and articulate the Division of Student Developments GA expectations and experience. Gain a better understanding of the assistantship offerings and experiences at Loyola University Chicago. Be able to select most desirable assistantship experience (specific department).

Accept an offer for a GA position at Loyola. Discerning whether Loyola is a good fit for the candidate.

Their Loyola GA experience matches their expectations as a result of attending Preview Days. Sharing their experience with their network of peers and professionals. Deeper level of investment and a stronger sense of community among GAs.

Assumptions: 1. Students will attend Preview Days and attend all applicable

workshops during Preiview Days 2. Prospective GAs will have little to no prior knowledge of Loyolas

GA experience and the city of Chicago 3. The programs chosen and presented will meet students needs and

help students discern institutional fit. 4. The Preview Days programs will be able to keep students interest

throughout the experience 5. University and Divisional partners are willing to take part in the

Preview Days experiencea

External Factors: 1. Prospective GAs expectations of the Preview Days experience 2. Number of attendees and cost of attendance 3. Availability of staff and resources (specifically monetary) 4. Weather and travel obstacles 5. Divisional and University power structure and politics 6. Institutional demographics (staff and students) 7. Existing interest in Loyolas programs and GA experience 8. Higher Education enrollment and application process 9. Reduction of assistantship offerings


Appendix B: Previous Preview Days Program Schedules

GRADUATE ASSISISTANT PREVIEW DAYS EVALUATION PLAN

38 Appendix C: Preview Days 2010 Evaluation Findings

Logistical Information

What is your current educational status? Current graduate student at Loyola 6 Do not plan on attending Loyola in Fall 2010 4 Incoming graduate student, Fall 2010 5 Still deciding on graduate school 9

How did you travel to Loyola?

I am local to the Chicago area and drove or took public transportation. 12

I drove to Loyola from out of town. 3 I flew into O'Hare/Midway. 8 Train 1

The logistical information (schedule, directions, map) was received in a timely manner. Agree 12 Disagree 1 Strongly agree 8 Strongly disagree 3

I feel that the logistical information provided prepared me for Loyola's preview day. Agree 14 Disagree 2 Strongly agree 8

Scheduling my interview(s) for March 3, 2010 was an easy process. Agree 10 Disagree 2 Strongly agree 12

I was well informed of my interview schedule. Agree 11 Disagree 2 Strongly agree 10 Strongly disagree 1

Comments

The timeliness of receiving the information about the two days could improve. The interview format was confusing and not explained, and the communication about the interviews was very poor. Many of us did not know if we were only interviewing with one or all of the assistantships we applied for, in addition to


39 who the main contact person was for the interviews and explaining that process.

I feel that the process of receiving responses was very confusing. We were given contradictory information as to when and how we would hear back from GA providers. This could be more clear in the future

Lodging

Lodging information was provided in a timely manner.

Agree 9 Disagree 3 Strongly agree 9 Strongly disagree 3

Did you stay with a student host or in Baumhart Hall?

No 22 Yes 2

Was the housing registration process effective?

Yes 2 Were you able to contact and sync up with your host(ess)/check in to Baumhart Hall in a timely manner? Yes 2 Was your host(ess)/residence hall accommodating during your stay? Yes 2 Did you know there was an option to stay with a current student/residence hall on Tuesday evening? No 11 Yes 11 Had you known, would you have utilized this option? No 9 Yes 2

Comments I was told not to count on student housing accommodations, and I did not receive any follow up once I

made an inquiry. I wish I would have heard further in advance because I made other arrangements. It would have been nice to have had internet access in Baumhart Hall.


40 School of Education Reception Did you attend the reception? No 11 Yes 13 Why didn't you attend? I had class during the reception time. 6 I was not aware of the reception. 1 I was still in transit to Chicago/Loyola. 4 The day of the reception worked well with my schedule (Tuesday) Agree 5 Disagree 1 Strongly agree 7 The time of the reception worked well with my schedule (4:30 p.m. - 6:00 p.m.) Agree 6 Disagree 1 Strongly agree 6 The length of the reception was sufficient to obtain the information I needed (1.5 hours) Agree 7 Strongly agree 6 The dinner following the reception was beneficial. Agree 7 Strongly agree 6 I had ample time to talk with Students Students Staff Faculty

Agree 4 6 8

Disagree 1 2 1

Strongly agree 8 5 4

Learning about the faculty and their research was beneficial to better understand the higher education program. Agree 6 Strongly agree 7


41

I felt welcomed at this event. Agree 4 Disagree 2 Strongly agree 7 *No comments provided for those that disagreed. Ironically, they agreed/strongly agreed with all other components of Tuesday evening. Please indicate your level of agreement with the following statements: - My experience on Tuesday left me with a good overall impression of the Loyola academic program. Agree 5 Disagree 1 Strongly agree 7

Comments:

Tuesday was wonderful! To improve it though, I wish there was a panel of student including some who did not have assistantships and how they were coping economically

It was a good night, highly informative and scheduled well. I did not realize the importance of attending the event on Tuesday evening. After attending other Preview

program, I believe meeting the professors and getting to know more a about the program could have impacted my decision on whether to attend Loyola. I also thought the Interview day would provide me with this information but sadly that was what the night before was all about.

I would have liked to know more about the class option before I arrived. I did not know about it and would have planned to attend had I known.

I would have liked to talk to more current students in the program. I thought it was an excellent way to start prior to the GA interview day following the reception. It provided

some opportunities to get to know a few of the other applicants, which helped reduce some nerves. I would continue to do a dinner reception in the future.

I felt we should have had more time to speak with current students. No one really moved around at the social; would it be possible to have dinner in one location and the social

in another? Prospective students just remained in their seats and it was really awkward to try to speak to a professor or staff member who was sitting at a full table.

Great help and made me feel at ease about the whole process and people in the program It was a great evening. It really provided me with a hands-on look at the academic program. This truly

enabled me to make a better overall judgment of whether or not the program was right for me.


42 Social Justice Class Did you attend the Social Justice class? No 20 Yes 4

By attending this class, I was able to gain a better

understanding of the higher education curriculum. Strongly agree 4

I felt welcomed in the class.

Strongly agree 4

I felt my time was well spent attending class. Strongly agree 4

Comments:

Loved it-loved the way the professor expected us to participate-and we did!


43 Wednesday: Interview Day

Please rate how effective the following logistical elements of interview day were:

Location (Lake Shore Campus) Effective 11 Ineffective 1 Very effective 12 Day of interview (Wednesday) Effective 12 Ineffective 4 Very effective 8 Interview time (e.g., 10:00 a.m., 11:00 a.m.) Effective 14 Ineffective 1 Very effective 9 Interview duration (30 minutes) Effective 13 Ineffective 1 Very effective 10 Length of day (8:00 a.m. - 4:30 p.m.) Effective 14 Ineffective 5 Very effective 4 Very ineffective 1 Room layout Effective 10 Ineffective 8 Very effective 5 Very ineffective 1 The following presentations were beneficial to my understanding of Loyola University Chicago Welcome (Dr. Rob Kelly) Agree 9 Strongly agree 15 Icebreaker Agree 4 Disagree 12 Strongly agree 6 Strongly disagree 2 What is Loyola and the Jesuit Mission? (Jake Jacobson) Agree 8 Disagree 1


44 Strongly agree 14 Strongly disagree 1 The Division of Student Development (Dr. Rob Kelly) Agree 11 Strongly agree 13 Moving to Chicago Information Session (Cliff Golz) Agree 16 Disagree 2 Strongly agree 4 Strongly disagree 2 Panel of Current Graduate Assistants Agree 18 Disagree 1 Strongly agree 4 Strongly disagree 1 I was able to efficiently fill my time between interviews. Agree 8 Disagree 6 Not applicable 1 Strongly agree 6 Strongly disagree 3 The amount/type of programming offered between interviews was sufficient. Agree 4 Disagree 10 Not applicable 2 Strongly agree 5 Strongly disagree 3 Comments:

A tour around campus, the student affairs offices, etc. A more structured way to meet/visit with everyone More on the classes, getting to know the prospective students as well as the current students in a more

familiar basis More structure for what do while waiting for interviews Campus tour! After going through this survey, I realized I signed up for interviews prior to receiving the schedule.

Because of this, I missed a session I would have preferred to attend, moving to Chicago. In the future it may serve the applicants better to see the structure of the day prior to being offered interviews. That way, applicants will see the structure/layout of the day and may be able to better accommodate their needs.

What to do if you do not have an assistantship For someone who is already a Loyola graduate student, none of the day's events aside from the interview

were helpful; however, I can see their relevance to those who are not already enrolled.


45 More information about the program - I am still trying to make sure I find a good fit school and program

wise. The dates of the reception and interview days were very inconvenient. I missed two days of class in order

to travel and attend the events. Holding these events later in the week, such as a Thursday or Friday, would be much more convenient than a Tuesday and Wednesday. I was not able to attend the panel of current students.

More interaction with current students and faculty. The conference room for us to sit in between interviews was too small and it was crowded. Great day. The presentation on What is Loyola and the Jesuit Mission? was quite overwhelming. It was too pushy

and religion focused. I really love and appreciate Loyola's mission, it's truly inspiring, but for non-Christian and especially atheists and agnostics the presentation was too strong. I believe it scared my non-Christian peer.

At a glance

Logistical information should be provided earlier for students to make flight arrangements and properly schedule interviews.

We should avoid scheduling sessions (i.e. Off-Campus living, grad panel) during interview times or hold them multiple times throughout the day. Students may have scheduled their interview not knowing about the session and then had to miss the session to interview.

We need to do a better job scheduling interviews for folks that have multiple interviews so they can be closer together. Maybe we can centralize this?

Providing a program for first-year Loyola students could be beneficial but does not appear to be required. Would be helpful to better communicate expectations to first-year students.

Offering lodging with current students would be helpful but not required. Finding graduate students to host students is very difficult.

The Tuesday evening reception was very well received! Only improvement would be to have people mingle more rather than staying seated. If current student, faculty, or staff were at a full table, it was difficult for prospective students to speak with that person.

The option to attend the Social Justice class was also very well received! Art did a fantastic job integrating the students into discussion and making them feel welcome.

Interview day: Overall well received. Improvement could be seen with the downtime students had. Offer campus tours, have graduate students on staff to speak with perspectives.

Could we hold the interviews in CFSU either in Bremner or in individual offices? Also, use the Hague as a gathering area for folks between interviews?

Can we offer campus tours or coffee with current grads during the downtime?


Appendix D: Preview Days 2013 Evaluation Instrument

Page - 2013 Division of Student Development Graduate Assistantship Preview Days Evaluation We would greatly appreciate your feedback on what we could do to improve the Preview Days experience. Please take a few minutes to tell us what you think!

Required answers: 0 Allowed answers: 0 Q1 Is Loyola the only Interview/Preview Day program for which you will participate? Yes[Code = 1] No (What other institutions have you/will you visit?)[Code = 2] [Textbox]

Required answers: 1 Allowed answers: 1 Q2 Which Preview Days session(s) did you attend on Sunday, February 24, 2013? (Check all that apply) Water Tower Campus Tour[Code = 1] Higher Education Orientation Dinner[Code = 2] Current Graduate Student Panel[Code = 3] Optional activities (please specify)[Code = 4] [Textbox] None of the above[Code = 0] [N/A]

Required answers: 1 Allowed answers: 5 Q3 Which Preview Days session(s) did you attend on Monday, February 25, 2013? (Check all that apply) Breakfast[Code = 1] Welcome, Introductions, Professional Development Opportunities[Code = 2] Overview of the Day[Code = 3] Jesuit Education 101: Jesu-What?[Code = 4] Moving to Chicago[Code = 5] Lake Shore Campus Tour(s)[Code = 6] Informal Social[Code = 7] Lunch[Code = 8] Social Justice Literacy[Code = 9] Grad Research: Scholar and Practitioner[Code = 10] Staff Panel[Code = 11] Lake Shore Campus Tour(s) #2[Code = 12] Optional Class: Student Development Theory[Code = 13] None of the above[Code = 0] [N/A]

Required answers: 1 Allowed answers: 14 Q4 Did you take advantage of the on-campus housing accommodations offered during Preview Days? Yes[Code = 1] No[Code = 2]

Required answers: 1 Allowed answers: 1

Next Page: Sequential


Page - 2 How would you rate the housing accommodations on the following components?

Q5 Information received regarding housing accommodations prior to your arrival Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q6 Check-in process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q7 Room provided Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q8 Check-out process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q9 Parking Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]



Display if Q4='Yes' Q10 Why did you not take advantage of the on-campus housing accommodations offered during Preview Days? [Code = 1] [Textbox]

Required answers: 0 Allowed answers: 1 Display if Q4='No'

Please rate the following aspects of your Preview Days experience:

Q11 Communication with DSDGrads Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q12 Information you received regarding Preview Days Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q13 Division of Student Development Graduate Assistantship Website Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q14 Online application process Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q15 Division of Student Development Staff Excellent[Code = 5] Good[Code = 4]


Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q16 Current Graduate Students/Assistants Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q17 Food selections Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]

Required answers: 1 Allowed answers: 1 Q18 Session locations Excellent[Code = 5] Good[Code = 4] Satisfactory[Code = 3] Poor[Code = 2] Unacceptable[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]


Next Page: Sequential Page - 3 Preview Days Sessions on Sunday, February 24, 2013 How valuable were the following aspects of your Preview Days experience?

Q19 Water Tower Campus Tour Extremely valuable[Code = 5] Very valuable[Code = 4] Moderately valuable[Code = 3] Not very valuable[Code = 2] Not valuable at all[Code = 1] Did not attend/Does not apply[Code = 0] [N/A]



Display if Q2='Water Tower Camp

Date post:	16-Nov-2015
Category:	Documents
Upload:	russellaivazian
View:	33 times
Download:	2 times

Aivazian-Preview Days Evaluation Project

Documents