Final Case Study: Methodology for Improving Alumni · PDF fileFinal Case Study: Methodology...

Final Case Study: Methodology for Improving Alumni Relations

ROBERT ROBINSON

December 9th, 2016

Com 564 Research Methods for Professionals

Washington State University

COLLEGE OF

ENGINEERING AND

ARCHITECTURE,

WESTERN STATE

UNIVERSITY


1

Table of Contents

Overview……………………………………………………………………………………..1-3

Focus Group Proposal………..………………………………………………………3-4

Survey Study Proposal……………………………………………………………….4-5

Experimental Study Proposal………………………………………………………...5-6

Focus Group Study…………………………………………………………………………..7

Moderator’s Guide……………………………………………………………………8-11

Discussion…………………………………………………………………………….12-13

Survey Study…………………………………………………………………………………14

Cover Letter………………………………………………………………………….14-15

Questionnaire…………………………………………………………………………15-22

Pretest Results…………………………………………………………………...........22-24

Discussion…………………………………………………………………………….24-25

Experimental Study…………………………………………………………………………..25

Design………………………………………………………………………………...25-27

Discussion.....................................................................................................................27-28

References…………………………………………………………………………………….28


2

Overview

The purpose of this case study is to develop and review methodology for improving

alumni relations at the College of Engineering and Architecture at Western State University.

This study was commissioned by the CEA staff members. They are interested in understanding

and improving relations with their 20,000 alumni. The specific goals include (1) evaluate the

effectiveness of current communication strategies and recommend preferred strategies of alumni

(2) strategies for increasing alumni donations. The department is nationally recognized for the

quality of its education and research. Specifically, the CEA representatives are interested with (a)

assessing how young adults value their education as a means of developing fundraising, (b) how

alumni appreciation of student experiences can be translated as a fundraising appeal, (c) how

targeted donation options for specific programs or problems might generate funds, (d) what are

the preferred media contacts between the department and alumni, (e) how to improve alumni

website contact, (f) how website content should be presented for maximum interest, (g) how to

message effectively to under 35 Alumni as a means of building a lifetime relationship.

A major focus in the development of this research methodology is the alumni. It is clear

they are a major part of the overall CEA-WSU community. Therefore the findings of the

proposed research will impact all stakeholders which include WSU administrators, CEA faculty

and staff, current graduate and undergraduate students, alumni, employers, public and private

donors. Finding more effective channels to message as a means to reach out to alumni may

influence how the CEA department communicates accomplishments and improve fundraising

appeals. The College has established a positive reputation for its quality degree programs and

innovative research. Many of the graduates work in similar industries and tend to remain in the


3

Pacific Northwest near major urban centers. Consequently, research may center on CEA alumni

who work in similar industries within specific geographic locations.

CEA fundraising officers have lingering questions about the best way to contact alumni.

Specifically, they question the benefits of e-mail versus regular mail fundraising solicitations and

the potential benefits of e-zines to replace expensive, hard-copy publications. Anecdotally, many

alumni seem to prefer infrequent contact with development staff members.

It is the purpose of this case study to investigate these issues through a mixed mode of

research approaches which includes qualitative research in the form of focus groups, quantitative

research through survey sampling and finally experimental research. The key components to

investigate include feedback on how to strengthen the CEA website. Specifically, identify

content areas that will resonate with alumni. Also, the CEA administrators request feedback on

the use of e-zines to replace print information. In relation to fundraising, they want to know

whether email or regular mail appeals for donations are preferred and how this compares to new

donation software on the website. This reflects a specific need to test and evaluate different

fundraising appeals. One of the key outcomes is to provide insights into the best way to develop

near term and long term donations with recent graduates. In order to achieve the desired

outcomes for the College of Engineering and Architecture three studies are proposed to generate

data upon which the department can build their growth strategies.

Focus Group Proposal

A key consideration in the generating feedback from alumni is to seek opinions. Focus group

methodology involves listening to the opinions of alumni in a small safe group through

discussion. It is proposed that a group of six to ten alumni participate in an open discussion with

a moderator. Using the resources of the alumni office a random sample of proposed participants


4

will be selected. The study will take advantage of the fact that most alumni work in similar

industries and live in the Pacific Northwest. Although focus groups represent qualitative

research with a limited sample, efforts will be made to include alumni representative of the

graduate population. One of the main characteristics important to this study is determining

opinions of under 35 alumni and alumni over that age. A purposeful sample will be recruited

through the WSU alumni relations office with consideration to age, industry, geographical

location, type of degree, income, race and current activity with alumni activities and donations. It

is proposed that six focus groups be selected; three under 35 and three over 35. Six participants

in each group will comprise a total of 18 participants under 35 and 18 participants over 35, with

a total of 36 participants. Letters of invitation will be sent to 100 alumni members above and

below age 35 who are located within similar geographical areas like Spokane, Tacoma, Olympia

or Seattle. A small stipend of $50 will be offered as an incentive for their time. In the letters of

invitation, there will be an opportunity to indicate preferred times for meeting, so that preferred

times can be considered.

Survey Study Proposal

The public relations staff at the CEA Department indicated that there are 20,000 CEA alumni

registered with the school. Currently about 20 percent donate to the department. The goal is to

reach at least a 35 percent level. In order to get a more complete picture of the issues, a survey is

developed to generate feedback. The key content areas covered in the targeted questions included

website evaluation, fundraising appeals and preferred methods of communication with CEA-

WSU interactions. The method used to generate the sample is based on a random sampling from

the alumni members. Since one of the research goals is to reach out to the younger alumni as

well as the older alumni, demographic responses in the survey will include gender, age (Under-


5

35 and Over-35), and geographical location. Using the total population of alumni at 20,000 it

would be estimated that with a 95% confidence level and a .02 confidence interval the sample

size should be 5,000. A randomly selected sample of 5,000 will be emailed the survey. With

survey return rates within organizations generally at 35% that would generate approximately

1,750 survey responses for analysis (Baruch & Holtom, 2008). A cover letter is also prepared to

highlight the importance of the survey to all alumni and to the College of Engineering and

Architecture. The questionnaire will include the key content areas of improving communications

with alumni through website evaluation, fundraising appeals and preferred methods of

communication with the department.

Experimental Study Proposal

An experimental study was designed to test the impact of a fundraising appeal on alumni over

age 35 and under age 35. The College of Engineering and Architecture is trying to build a

motivating connection with the under 35 alumni as a means of building a future fundraising

relationship. Survey research by the department indicates there is enthusiasm to give among

donors under 35 years of age. It is suggested that the younger donors have the highest motivation

but are likely to have the least capacity to give. In order to examine attitudes and response

behaviors two participant ages will be studied: alumni over 35 and under 35. The design of the

video fundraising appeal will be developed based on the feedback from the focus group and

survey research studies. The goal of the department is to increase donations from 20 percent to

35 percent. Therefore, evaluating the impact of the fundraising appeals is essential to achieving

that goal. Departmental research cites the fact that under-35 alumni are willing to give back, but

within their budget. Appeals to diversity and global consciousness are motivating and should be

considered within the fundraising appeal.


6

The independent variable will be a video fundraising appeal which includes department

chairperson explaining the importance of donations to the success and leadership of the

department. They will highlight current research, student projects and how alumni have made a

difference. It will end with a link to online donations. Experimental Group 1 (over 35) will

receive exposure to the fundraising video. The Control Group 1 will not receive the treatment.

Both groups will receive a post-test attitude measure to determine differences between alumni

who watched fundraising appeal compared to those who did not. The post-test attitude

instrument will be the dependent variable. Group 3 Experimental participants under 35 will be

presented with the same fundraising appeal. Group 4 Control participants will not view the

fundraising appeal. As with Groups 1 and 2, 3 and 4 will complete the same dependent variable

using the post-test attitude measure. A between group design will be used in which n = 100

randomly sampled alumni within the above and below age framework will be assigned to one of

the four groups. With n = 100 in each group, that would comprise a total of N = 400.


7

Focus Group Study

Focus Group Letter of Invitation

College of Engineering and Architectures

Western State University

Alumni Association

College of Engineering & Architecture


Pullman, Washington, 99164

[Date]

[Name]

[Address]

[City, State, Zip]

Dear WSU College of Engineering and Architecture Alumnus,

As a graduate of the College of Engineering and Architecture at Western State University

we would like to invite you to attend a focus group discussion. The purpose is to help us

improve our communications with alumni, share our department innovations and

accomplishments and encourage fundraising appeals. One of our primary concerns is how you

evaluate the current website in terms of effective communication and outreach. The session will

be December 3rd

, 2016 from 6:30 to 8:30. The focus group will provide you with an opportunity

to find out what other graduates think about how to improve our outreach to alumni.

Your opinion will provide valuable insight to the department. The department has

contracted with the consultant firm of Robinson Research Associates. They will moderate the

discussion and analyze the results. Your responses will remain anonymous, only themes and

ideas will be reported on the department website when the results are completed. This study is

part of a longer evaluation process. Your feedback will provide a valuable starting point for our

study. If you would like to take part in the focus group on December 3rd

, 2016, please sign the

reply box at the bottom of this letter and email it back to us. You will receive a $50 gift card for

your participation. If you received a mail copy, please return it in the pre-paid return envelope.

If you have any questions, please contact the department at 206-xxx-xxxx.

Sincerely


8

John Smith

Director of Alumni Relations

[email protected]

(xxx) xxx-xxxx

---------------------------------------------------------------------------------------------------------------------

Yes I will be glad to attend the Focus Group Session

No, I will not be able to attend

Contact information”

Name_________________________________________________

Email address__________________________________________

Phone:________________________________________________

Focus Group Moderators Guide

A moderator’s guide is included to guide the participant’s discussion and to generate information

that will help the CEA representatives make decisions on how to guide alumni participation and

fundraising appeals. The goal of the focus group interactions is to provide a structure that will

facilitate open discussion.

Introduction and Welcome:

Welcome and thank you for participating in this CEA alumnus focus group.

Introduction:

I would like to introduce myself. My name is _________ and I am the moderator for today’s

discussion. I would like to present ________who is our assistant moderator. They will be

recording our responses so that we may learn from them.

Purpose of the Focus Group:

We have been asked by the public relations department of College of Engineering and

Architecture at Western State University to seek your opinions about alumni relations. We

appreciate your willingness to participate and value your input. As moderator I will work to

guide our questions with follow-up to focus on your responses. The opinions you share with us

today will help guide future planning for CEA alumni interaction and communication.

mailto:[email protected]


9

Specifically we are looking for ways to increase alumni interaction with the website, increasing

online communications with alumni and what fundraising appeals might be attractive. You were

randomly selected for this focus group. We centered in on (geographical area), as a large

representative sample of graduates are located here. Although not required by the University

Institutional Review Board, we ask that you review and sign the consent form for our records.

We need your input and want you to share your honest and open thoughts with us so that we may

better meet your needs as alumni.

Ground Rules:

This focus group will be about two hours long.

We want you to do the talking.

o We would like everyone to participate

o We may call on you if we have not heard anything from you

Remember, there is no right or wrong answer.

o Each person’s opinion is important

o Please share with us even if you agree or disagree

o We want to learn what you are thinking

Your opinions will remain anonymous

o We want you to share freely knowing that what is said in this room stays here

We will be tape recording our discussion

o We want to capture the responses, but we don’t identify anyone by name.

Please turn off all cell phones

Speak clearly one at a time

Are there any questions?

Icebreaker:

Let’s go around the room and briefly introduce ourselves.

Briefly share with us what you value most about your education at the College of

Engineering and Architecture at Western State. What do you appreciate about your

education and experience at CEA-WSU?

Key Questions:

The questions prepared today are designed to tap into your thoughts about the alumni website,

fundraising appeals and preferred methods of communication. I will present each one


10

individually. We will spend about 25 minutes discussing the topic. When we are done we will

move on to the next question.

Topic: Website (Use computer to link to website on large screen. Present the landing page.)

Moderator: Most of you are familiar with the CEA-WSU website. Have a look at the landing

page.

1. How many of you have visited the alumni website in the last year? Show of hands.

2. For those who visited the website, how long did you spend on the website?

Was there anything specific you were looking for?

3. Was the content relevant to your interests?

4. If it was not, why not?

5. Those of you who did not visit the website, what were the main reasons?

6. What would encourage you to visit the website?

Fundraising Appeals: Present current website fundraising appeal

Moderator: Thank you for your website discussion and feedback. We now want to move on to

your ideas on improving fundraising appeals.

Moderator will share with the Under 35 Alumni Focus Group: Survey research results

indicate enthusiasm to give among donors under 35 years of age. These younger donors

have the highest motivation but are likely to have the least capacity to give. Try to

consider how this issue applies to you as you answer and discuss the questions.

Moderator will share with the Over 45 Alumni Focus Group: Giving to education

increased to $41.33 billion in 2012, an estimated 7 percent increase (not adjusted for

inflation). Four-year institutions receive roughly 75 percent of the donations in this

category. We would like you to consider ways to improve our fundraising appeals as you

answer and discuss the lessons.

1. How many of you have ever donated to the CEA College at Western University? Show of

hands.

How many of you have donated in the past year?

Can you identify how you were motivated to donate?

Do you feel your donation will make a difference?


11

2. Are there any specific programs or ideas you would be motivated to donate to within the

school?

3. Would you like to have a choice of donation options so you can designate where you

want your money to go?

4. Would you like feedback on the impact of your donations within the CEA-WSU?

5. The department website is now implementing an online donation form to streamline the

process. Currently, it is through the mail. Do you think this will improve opportunities

to donate?

Preferred Methods of communication

1. What is your main channel of communication with the CEA college?

2. How many of you have visited the website in the last month, last year? Show of hands.

3. Does it provide you with the information you are seeking or would like to know more

about?

4. As you all know the CEA-WSU Department is well known for its excellent education and

outstanding research. They want to share with alumni their accomplishments.

What is your preferred method for receiving updates?

Would you like updates through email, e-magazines, Facebook, or other social

media?

5. How often would you expect updates and messaging? When would you consider

messaging to be too often? What do you consider a balance?

6. What type of messaging to encourage fundraising would you consider appealing or

motivating to you?

Ending Questions/General Discussion:

I want to take this time to thank everyone for participating in today’s focus group. As we end the

session, is there anything else that you feel is most important to this discussion? Have we missed

anything?

Conclusion:

I want to thank everyone for coming today. We appreciate your time and participation.


12

Focus Group Discussion:

The data derived from the focus group will provide an in-depth look into the current ideas of

alumni Above 35 and Under 35. The key content areas covered in the targeted questions for

discussion included website evaluation, fundraising appeals and preferred methods of

communication with CEA-WSU interactions. It is the goal of the focus group research to

uncover information about these issues that might not be discovered in survey or individual

interviews (Austin & Pinkleton). The social interaction of the alumni group should work to

stimulate shared observations. It also provides an opportunity to probe positive and negative

responses. In this way, the results provide good face validity in the sense that they can assess

what is said with the context of what was meant by the participant. The moderator can probe to

confirm and clarify responses. However, since the groups are small, the insights gained may not

be translated to generalized conclusions. The sample was not selected randomly. It is non-

parametric with low predictability. The purpose of the focus groups in this study is to explore

alumni insights, examine themes across statements, dig deeper into these issues and generate

some ideas and themes that may offer some possible strategies to consider.

The characteristics of focus groups represent qualitative responses. They provide an opportunity

to get feedback from the respondents about the characteristics that influenced how the ad

campaign. Focus groups are generally much smaller than quantitative survey research. Sample

size is generally not an issue in focus groups as it is not designed to draw definitive conclusions.

In other words, it is not possible with a limited sample to project the findings as representative of

the larger population. Therefore, the small sample size limits external validity. Also each of the

four focus groups, even when presented with similar questions may follow different lines of


13

thematic discussion. Therefore there may be less consistency or reliability with this type of

sampling.


14

Survey Study

Survey Cover Letter



Alumni Association

College of Engineering & Architecture


Pullman, Washington, 99164 [Date]

[Name]

[Address]

[City, State, Zip]

Dear WSU College of Engineering and Architecture Alumnus,

Our college of Engineering and Architecture at Western State University is respected for

its achievements in research and teaching. Many alumni have told us of the value of their

education and how it has helped their careers. One of our goals is to improve our

communications with alumni. We are conducting a survey of alumni to give us feedback on how

we can serve you more effectively. The purpose is to help us improve our communications with

alumni, share our department innovations and accomplishments and encourage fundraising

appeals. The key content areas covered in the targeted questions included website evaluation,

fundraising appeals and preferred methods of communication with CEA-WSU interactions. In

the last year we have observed many alumni visit our website but don’t stay long to read the

information. With your feedback we may find better ways to meet your needs. We have

contracted with Robinson Research Associates to develop and analyze the survey. This is part of

a continuing mixed-method approach to study the problem. When the study is completed we

will share the findings on our website.

Your participation in the survey will be completely confidential and data will be reported

in general summaries in the written reports. This request has been sent to you by email. We can

mail you a copy if you would prefer. It should take no more than fifteen minutes to complete.

Once you return the survey, your contact information will be removed. Your participation is

voluntary.

We want to thank you for participating in our survey. Your insights and opinions will

help our alumni organization provide more meaningful communications. Not only will you

provide guidance in improving our website design abut also guide us in fundraising appeals.

Please call us at 206-xxx-xxxx if you have any questions. There is a three week period for all

alumni selected to return the survey. The final return date will be xx/xx/xxxx. Upon receipt of


15

the survey we will send you an email VISA prepaid gift card for $10 in appreciation for your

time. When completed, return by sending back through email.

Sincerely, [Name]

Director of Alumni Relations

College of Engineering and Architecture

Survey Questionnaire


Western State University Alumni Survey

Introduction

We would like to thank you for participating in our alumni survey. The purpose of our survey is

to generate feedback from our alumni so that we may better serve you. Specifically we are

looking for ways to improve our communications with alumni through our website. We value

your opinion about the best ways to share our department innovations and accomplishments and

encourage fundraising appeals. Give us your feedback on topics, events and programs that

interest you. Please provide us with your honest opinions so that we may improve our alumni

outreach. The average time to complete this questionnaire is approximately 15 minutes. Mark

the questions with an x to indicate your opinion or choice.

College of Engineering & Architecture Experiences

1. What was your major focus of study

Bioengineering

Chemical engineering

Civil engineering

Computer engineering

Electrical engineering

Materials science & engineering

Mechanical engineering

Software Engineering

Construction Management

Infrastructure Engineering

Interior Design

Landscape Architecture

Structural Engineering

Other__________________


16

2. How satisfied are you with your overall college experience at WSU?

Strongly Satisfied Strongly

Dissatisfied

1 2 3 4 5

3. How satisfied are you with the College of Engineering and Architecture experience

at WSU?

Strongly Satisfied Strongly

Dissatisfied

1 2 3 4 5

4. Please indicate how you rate your continuing activity with the College of

Engineering and Architecture at WSU.

Hardly Ever Occasionally Sometimes Frequently Often

How often do

you participate

in Alumni

Activities?

5. How would you rate alumni outreach experience?

Effective Ineffective

1 2 3 4 5

The College of Engineering & Architecture Website

6. How many times have you visited the CEA-WSU website in the last year?

None

1-3

4-6

More than 6

7. How would you rate the overall impression of the website?

Ineffective Effective

1 2 3 4 5


17

8. Mark the type of information you were seeking. Mark as many that apply.

New research and innovations by CEA-WSU

Department Initiatives

Student accomplishments

Alumni accomplishments

Alumni Activities - Contacts

Donations

Other_______________

9. How would you evaluate the quality of information you were seeking on the

website?

Excellent

Very Good

Satisfactory

Poor

Unacceptable

10. Please mark all content areas on the website that you would like to see improved.

Mark as many that apply.

News about current research led by professors in the department

Current news about goals and department objectives

Information about student accomplishments

Information about alumni accomplishments

Donations that have made specific research and support possible

Alumni activities locally and around the country

Other_______________________

11. What additional features would you like to see integrated into the website to

improve interest? Mark all that apply.

Expanded links to social media like Facebook and Twitter

Highlighted news with video

Opportunities to respond and interact to department news stories

New programs

More multi-media presentations

Other_____________________________


18

The College of Engineering & Architecture Preferred Channels of

Communications

12. What is your preferred channel of communication with the alumni office?

E-mail

E-Newsletter

E-magazine

Print Newsletter

Print magazine

Facebook

Twitter

Website

Other_______________________________

13. What is your preferred channel of communication with the CEA-WSU updates?

E-mail

E-Newsletter

E-magazine

Print Newsletter

Print magazine

Facebook

Twitter

Website

Other_______________________________

The College of Engineering & Architecture Fundraising Appeals

14. How often have you donated to CEA-WSU since graduation?

0

1-3

4-6

More than 6 times

15. How would you rate your overall donations to all organizations in the last year?


19

$0

$10 - $20

$30 - $40

$50 - $60

$70 - $80

$90 - $100

$101 - $500

Above $500

16. How have you received fundraising requests?

E-mail

Fundraising

Newsletter

Annual giving campaign

Phone call

No contact

17. How would you rate current fundraising appeals at the CEA-WSU website and

communications with you?

Outstanding

Better than expected

Meets expectations

Unremarkable

Poor

18. What improvements would you suggest in our fundraising appeals? Mark all that

apply.

Produce multi-media to illustrate new

research and innovations

Produce multi-media presentations of

faculty at leading conferences

Produce multi-media presentations of

outstanding students

Offer alumni memorabilia with

donation requests

Provide more fundraising events

Produce multi-media presentations and

interviews with outstanding alumni

Produce multi-media stories on alumni

activities

Provide alumni with choices in what

program they would like to make

donations.

Provide donation links on the website.

Donation opportunities to support

scholarship


20

The College of Engineering & Architecture Demographic Information

In order to help us understand our alumni response patterns, please indicate what characteristics best

represent you. We wish to reassure you that all information will remain confidential and will not be used

for any other purpose.

19. Please confirm that you are a graduate of the College of Engineering and Architecture at Western

State University

Yes

No

20. What year did you graduate?

______________________

21. What is your gender?

Male

Female

Other

22. Please indicate your race/ethnicity.

American Indian or Alaskan Native

Asian

Native Hawaiian or Other Pacific Islander

Black or African American

Hispanic or Latino or Spanish Origin of any race

White

Two or more races

Race and Ethnicity unknown

23. Please indicate your marital status.

Single

Married

Separated

Divorced

Widowed

Other

24. What age range are you in?

Under 35


21

36-39

40-50

60+

25. What is the highest level of education you have completed?

Bachelor’s Degree

Master’s Degree

Graduate or Professional Degree

Doctoral Degree

26. Please indicate the range of your personal yearly income before taxes during the past 12

months?

Less than $25,000

$25,000 to $34,999

$35,000 to $49,999

$50,000 to $74,999

$75,000 to $99,999

$100,000 to $149,999

$150,000 or more

Prefer not to answer

27. Describe your current employment status

Full-time employment

Part-time employment

Self-employed

Unemployed

Looking for employment

Retired


22

Military service

Other_________________________

28. Please add any other comments or thoughts to help us improve our alumni outreach

Pretest Results

The preliminary cover letter and survey questionnaire were presented to six college graduates.

Time taken to complete the survey was recorded. On the average it took approximately 10-15

minutes. There were a total of 26 questions. The key content areas included feedback on the

evaluation of their college experience and continuing activity, interactions with the College of

Engineering and Architecture website, preferred channels of communication, fundraising appeals

and demographic information. Feedback from respondents was used to modify cover letter and

questionnaire.

Cover Letter

Feedback was generally positive for the cover letter. There was consensus that the key

points were clear.

o That the college of Engineering and Architecture at Western State University is respected

for its achievements in research and teaching.

o Many alumni have told us of the value of their education and how it has helped their

careers. One of our goals is to improve our communications with alumni.


23

o The purpose is to improve communications with alumni, share department innovations

and accomplishments and encourage fundraising appeals.

The key content areas covered in the targeted questions included website

evaluation, fundraising appeals and preferred methods of communication with

CEA-WSU interactions.

In the last year many alumni have visited our website but don’t stay long to read

the information.

Feedback will help the department find better ways to meet your needs.

o Participation in the survey will be completely confidential and data will be reported in

general summaries in the written reports that will be shared with alumni.

o The request has been sent by email. There is an option to receive the survey by mail.

o It should take no more than fifteen minutes to complete. Email information will be

deleted from the survey. Your participation is voluntary.

Questionnaire

Feedback on the introduction to the questionnaire was considered to be succinct and to

the point. It states the purpose about the importance of obtaining feedback to better serve

alumni.

It states that the feedback is targeted to communications with alumni and effectiveness of

the website. It seeks honest opinions about their experience at the College of

Engineering and Architecture. It also requests information about fundraising appeals.

It states clearly that the questionnaire should take no longer than 15 minutes.

The primary mode of questionnaire response is through email. It is requested that

respondents mark an X on the appropriate response. Pretest takers requested a statement

about completing questionnaire and returning to alumni office. This change was made.


24

It was suggested that question 18 should be changed from “What would you suggest as

improvements in fundraising appeals?” to “What improvements would you suggest in our

fundraising appeals?”

Demographics

Three participants noted that there was no marriage status or race/ethnicity. Review of

current recommendations by the Washington State Department of Health guided the

categories (VanEenwyk, 2010). These categories were added to the questionnaire.

Four participants requested in Question 26 that when answering income range, there

should be the option of “preferring not to answer.”

Survey Discussion

This survey designed for the public relations staff at the CEA Department will be administered to

randomly sample of 1,750 alumni of which there are 20,000 CEA alumni registered with the

school. The survey will be emailed and self-administered along with a cover letter detailing the

survey. The purpose of the research is to generate feedback on current departmental

communications and fundraising appeals. The current estimation is that only about 20 percent

donate to the department. In order to reach at least a 35 percent level the department has to

develop a more complete picture of the issues. The key content areas covered in the targeted

questions included website evaluation, fundraising appeals and preferred methods of

communication with CEA-WSU interactions. Findings will be analyzed separately by under 35

and over 35 age demographics.

The sample generated for this study provides a relatively accurate assessment of the

alumni population from which a degree of external validity can be generated. The pre-testing of


25

the survey provided feedback on the internal validity of the questions. At face value, the general

question content appears to be representative of the departmental objectives. The revision was

valuable in developing the final survey questionnaire. The questions were relatively consistent

and should offer a format and a reliable survey format. The results will be summarized by

question type and analyzed for alumni preferences. One of the challenges is obtaining an

adequate return rate (Fink, 2017). Surveys generally have a low response rate. With the cover

letter, there is incentive to return the survey in which participants will receive a $10 email gift

card. This may work to improve return rate which will result in a more representative sample of

alumni.

Experimental Study

An experimental study was designed to test the impact of a fundraising appeal on alumni over

age 35 and under age 35. The College of Engineering and Architecture is trying to build a

motivating connection with the under 35 alumni as a means of building a future fundraising

relationship. The department observed that generally survey research indicates enthusiasm to

give among donors under 35 years of age. These younger donors have the highest motivation but

are likely to have the least capacity to give. In order to examine attitudes and response behaviors

two participant ages will be studied: alumni over 35 and under 35. The design of the video

fundraising appeal will be developed based on the feedback from the focus group and survey

research studies. The goal of the department is to increase donations from 20 percent to 35

percent. Therefore, evaluating the impact of the fundraising appeals is essential to achieving that

goal. Departmental research cites the fact that under-35 alumni are willing to give back, but

within their budget. Appeals to diversity and global consciousness are motivating and should be

considered within the fundraising appeal.


26

The independent variable will be a video fundraising appeal which includes department

chairperson explaining the importance of donations to the success and leadership of the

department. They will highlight current research, student projects and how alumni have made a

difference. It will end with a link to online donations or request for mailed forms. Experimental

Group 1 (over 35) will receive exposure to the fundraising video. The Control Group 1 will not

receive the treatment. Both groups will receive a post-test attitude measure to determine

differences between alumni who watched fundraising appeal compared to those who did not.

The post-test attitude instrument will be the dependent variable. Group 3 Experimental

participants under 35 will be presented with the same fundraising appeal. Group 4 Control

participants will not view the fundraising appeal. As with Groups 1 and 2, 3 and 4 will complete

the same dependent variable using the post-test attitude measure.

A between group design will be used in which n = 100 randomly sampled alumni within the

above and below age framework will be assigned to one of the four groups. With n = 100 in

each group, that would comprise a total of N = 400. If pre-tests were used, then the experimental

design would be increased to 8 groups with a total of 800 participants. Pre-tests can work as

intervening variables by alerting participants to the content messaging. The department has the

option of expanding the design if it chooses. In the proposed design, it is recommended that two

experimental groups above and below 35 receive the fundraising appeal. The control groups will

receive the same post-test to see if there a significantly different response between groups. In

other words, does the fundraising appeal make a significant difference in alumni responses. A

total random sample of 400 will allow for sufficient data to determine the impact of the

fundraising appeal. This design can be replicated as needed for additional evaluation.


27

Between Groups Design

Groups Random Sample Fundraising

Appeal

Post-test

Group 1 (Alumni

Experimental +

35)

R n = 100

X

01

Group 2 (Alumni

Control +35)

R n= 100

02 Group 3 (Alumni

Experimental- 35)

R n = 100

X

03 Group 4 (Alumni

Control - 35)

R n = 100

04

Discussion

In this design, one treatment condition across two experimental groups with two levels by age

will compared responses on an attitude measure towards donations with two control conditions

by age. This will allow researchers to assess the effects of the independent variable, which is the

fundraising appeal. It is recommended that the attitude towards fundraising measure be assessed

for content validity. Multiple questions should be designed to comprise multiple item indexes

related to the fundraising appeal (Austin &Pinkleton, 2015). They can be pre-tested for

reliability prior to use. The indexes can be evaluated through an item analysis to determine

relatedness to the key questions: 1. Importance of fundraising, 2. Effectiveness of fundraising

appeal presentation, 3. Ad impact on personal donations, 4. Suggestions for improvement.

This between groups design combines a treatment and control group with random assignments of

participants. The sample size and assignment will work to increased reliability and external

validity. It is designed to demonstrate that differences between treatment and control group are

due to chance. With 100 participants randomly selected within each of the four groups it

decreases selection bias of pre-existing group effects. Random assignment also works to increase

external validity which represents generalizability of results to the larger alumni population.


28

Taken together, the focus group study, survey study and experimental study will provide

valuable feedback to the College of Engineering and Architecture on how to improve

communications and relations with alumni in achieving their goal of increased donations.

References

Austin, E. & Pinkleton, B. (2015). Strategic Public Relations Management. New York,

Routledge.

Baruch, Y. & Holtom, B. (2008) Survey response rate levels and trends in organizational

Research. Human Relations, 61; 1139-1160. Retrieved from

http://hum.sagepub.com/content/61/8/1139.abstract

Fink, A. (2017). How to conduct surveys. Los Angeles, Sage.

VanEenwyk, J. (2010) Revised guidelines for using racial and ethnic groupings in data

analyses. Washington State Department of Health. Retrieved from

http://www.doh.wa.gov/portals/1/documents/5500/raceethnguidelines.pdf

Date post:	06-Mar-2018
Category:	Documents
Upload:	dinhthu
View:	215 times
Download:	2 times

Final Case Study: Methodology for Improving Alumni · PDF fileFinal Case Study: Methodology...

Documents