DAVID T. GREEN Washington State University Fall 2016
COM564 RESEARCH METHODS FOR PROFESSIONALS
FINAL CASE STUDY
Background………………………………………………………………………………………….1
Overview of the Focus Group……………………………………………………………1-2
Overview of the Survey…………………………………………………………………….2-3
Overview of the Experiment………………………………………………………………..3
Focus Group………………………………………………………………………………………..4
Research…………………………………………………………………………………….4
Set Up…………………………………………………………………………………………4
Moderator’s Guide………………………………………………………………….5-7
Reliability and Accuracy……………………………………………………………..8
Cover Letter………………………………………………………………………………………..9
Survey…………………………………………………………………………………………. 10-13
Pretest Results……………………………………………………………………..14-15
Reliability and Accuracy……………………………………………………………16
Experiment……………………………………………………………………………………….17
Design………………………………………………………………………………………17
Function…………………………………………………………………………………..18
Survey…………………………………………………………………………………18-20
Reliability and Accuracy……………………………………………………….…..21
Conclusion………………………………………………………………………………………..22
1 | P a g e
We have been hired by the College of Engineering and Architecture at Western State
University to design a comprehensive research program to help CEA public relations staff
members communicate more effectively with College alumni. Our goal is to be able to help
development staff members increase the effectiveness of CEA communication and fundraising
efforts.
Many CEA graduates end up employed in similar industries and a high percentage chose
to remain in the Pacific Northwest. As a result, they tend to live in or near the state’s urban
centers and a significant number end up working for large firms with engineering or related
functions as a core part of their business. As a result, there is potential to reach out to CEA
alumni who work in specific industries, who are employed by key corporations and/or who live
in specific geographic regions of the state.
Our plan is to use focus groups from three major urban areas in the state of Washington.
The cities picked for this study are Seattle, Spokane, and Vancouver. We feel that the need for
focus groups is vital in determining the accuracy of the previous observations made by the CEA.
We already have learned about the alumni’s attitudes and actions towards their charitable
contributions to the Western State College of Engineering and Architecture. However, it is
2 | P a g e
imperative that there are research gaps that need to be addressed, meaning there are still
questions to be answered to make sure that the study is accurate in dissecting the strengths and
weaknesses of the approach that the current public relations group is involved in.
As mentioned earlier, these focus groups will take place in Seattle, Spokane, and
Vancouver. We will use a random sample from the list of CEA alumni living in these urban
areas. This way we can learn more about such topics as demographics, previous contributions,
and even more importantly, we can get a better idea about how the alumni feel about the College
of Engineering and Architecture. We are also looking to see if/how alumni are currently involved
with the alumni association.
One of our major goals with this study is to determine which approaches are working
with alumni, which are not working, and if there are any additional approaches that they would
like included. To get this information, we will develop a quantitative survey presented to CEA
alumni from across the United States. We want a sample size of 750 CEA alumni with the
assumption that we will receive approximately 600 responses (See page 16 for calculations). We
will use a stratified random sampling of CEA alumni from across the country. We will not
include any alumni who have international addresses, due to time and expense.
3 | P a g e
The accompanying cover letter will be presented in a friendly tone and kept as short as possible,
while giving the prospective participant all of the information they will need regarding this
survey.
Our experiment will look at how likely an alumnus will give to the CEA if he/she is kept
up to date and given more information about the alumni association than those who are given
basic information. What we will look at here is if the graduates’ knowledge of the exciting things
going on with the CEA will facilitate them to give back to the school in the form of donations.
The experiments will take place in all three target cities of Seattle, Spokane, and Vancouver. We
will be using twelve participants. All participants will be shown a video, talking about the alumni
association in general, how the association functions, the association officers will be introduced,
and the group will be given a general idea of how donations can help. After this, the participants
will be given a 15 minute break and then split into two groups of six. The control group will
continue to learn general information about the association. The test group will be visited by
members of the board of the association who will delve into much more pertinent information,
such as the specifics about how the donations are being used and what benefits the alumnus can
receive from giving back to the department. They will also take questions from the participants.
At the end of their meetings, both groups will be asked about their inclination to donate. The idea
here is to find out if learning more about how the donations are being used and two-way
conversation with members of the alumni will make them more inclined to donate to the
department.
4 | P a g e
When we look at the information presented to us, we can focus on age being a major
factor in whether an alumnus donates, and how much they donate. “As they age, many alumni
recognize their WSU education as a critically important aspect of their personal development and
one of the most significant personal investments they made as young adults.” This can be a great
place to start when assembling our focus groups.
The Research:
Therefore, we will conduct six focus groups, two each in Seattle, Spokane, and
Vancouver. We will use the CEA alumni list to gather 6-12 former CEA students. We will split
the groups up by age in each city, with one group being those who graduated between 1995 and
2016, and one group from those who graduated before 1995. For these groups, we want to use
our demographic search wisely. We want an equal number of male and female participants. We
will conduct these focus groups in hotel meeting rooms in the three cities.
Setting up for the Focus Groups:
We want the participants of our focus groups to be welcomed at the sessions by having
snacks set up on a table, plus letting them mingle a while with other members of their alumni. At
each seat, we will have bottled water available. At each seat will be consent forms which are to
be read and signed by all participants before the focus group begins. Each focus group will be
filmed and shown to other members of the research staff, live via closed circuit television.
5 | P a g e
The Moderator’s Guide to the Focus Group:
Joe Smith (Moderator):
I want to welcome you to this focus group, brought to you by the College of Engineering
and Architecture. As you probably already know from reading the letter we sent you by mail, this
discussion will focus on the various methods of communication with the CEA and the alumni
association. We will be discussing the CEA’s website, magazine and fundraising efforts. Our
goal is to learn about how your experience has been so far with the association and to get some
feedback on what it is that you would like to see improved with the methods of communication
used to keep you in touch with the CEA. This is to be an open and friendly discussion and I want
to tell you up front that there are no right or wrong answers. Please speak freely and let us know
your honest opinions. If you feel the need to leave the room at any time, please let me know and
there will be no problem with this. Are there any questions that you may have before we begin?
Okay, let’s get started. I want to first ask you about your current interactions with the
CEA. Look up hear at the flip chart and when I talk about a method of communication that you
use, please raise your hand, highly please. Do you ever read about CEA from a magazine? The
CEA alumni monthly newsletter that is mailed to you? The CEA alumni website? Any CEA
social media page, i.e. Facebook, Twitter, etc.?
Next I want to ask you this: Do you receive any information from the association? By
this, I am asking, “Are you on any mailing lists?” Do you keep the association informed if you
change your address, or any other contact information?
Now, let’s take a look at the CEA’s social media sites. First, we will look at the website
(Showing live website up on the screen). As I scroll through the home page, are there any initial
6 | P a g e
thoughts that come to mind? This may be a good time to ask – Have you ever been to this site
before? If not, why haven’t you visited this site?
Now we will take a look at the CEA’s social media sites (shows live feeds from social
media sites). Which of these peeks your interest? Do you find any of these unappealing? Why?
After viewing the website, and social media sites, which one(s) would make you want to
find out more about?
What is it about either the website, social media, or the magazine do you enjoy? Are there
certain elements in any of these sites that you like more than others? What could be missing from
these sites that you feel could be of importance to a member of the CEA alumni?
Do you know who is on the board of the CEA alumni association? If not, would you like
to know who they are and learn about them?
Let’s move on to your current connection with the association. Studies have shown the
approximately 20 percent of the 20,000+ CEA alumni donate to the association. By a show of
hands, do you currently donate? Have you donated in the past? Remember, all of this is
confidential and will not be published. What is it that gets you to donate? Is there a particular
event or story that gets you excited about donating?
For those of you that do donate – let’s talk about how it is that you go about donating. Do
you send in a check? Do you donate online? Do you have donations automatically sent in each
month or year?
7 | P a g e
For those of you who have not donated yet – is there a reason you don’t donate? Is there
something that you can think of that the association could do to make it so you felt good about
supporting the CEA?
Finally, are you happy with the communication you receive from the CEA? How do you
think it could be improved?
I want to thank you all for joining us here tonight. The information and ideas that you
have provided are vital for the CEA.
8 | P a g e
Due to the fact that the questions were set up to elicit responses dealing with the
participant’s communications with the CEA, we can expect acceptable internal validity. As a
rule, reaching good, even acceptable, external validity using focus groups is a problem. And with
this focus group, we have to say that the external validity would be rated as low, at best. Do the
alumni who live in these three urban areas represent the larger population of the state that does
not live in urban areas? Do alumni that have moved away from the state have different ideas
about their communication with the association? One of the reasons we focused on age in this
study was to see if there were any differences in the way they looked at communication. Would
the older group prefer their information by U.S. Mail? Would the younger group ignore the same
style of communication? The best way to assess this kind of focus group is to take the ideas they
give and incorporate them into a broader survey. Validity aside, these groups give us some
valuable feedback and are useful for our study in this case.
9 | P a g e
College of Engineering and Architecture
Western State University
P.O.Box 714
Pullman, WA 99163
Alumna
123 State Ave.
Your Town, WA 98642
Dear Alumnus,
The College of Engineering and Architecture would appreciate it if you would take the time to
complete the enclosed survey. Your feedback is very important and is valued by the CEA in our
continuing efforts to improve the communication between the CEA and its alumni.
It is extremely important to us here at the CEA to bring you important and timely information
about the school. With the results we gather from this survey, we can keep you up to date on
events, innovations, changes that happen within the school, and much more. We want you to be
an active part of the CEA family, and that means we need your input to make this happen. Your
individual answers to this survey will be completely confidential.
Please do not hesitate to contact us if you have any questions about the survey, as we want you to
be a big part of the future of the CEA.
Thank you in advance for participating.
Sincerely,
David T. Green
Public Relations and Communications Manager
Western State University
10 | P a g e
Demographics
Please answer all questions by checking the box next to your response:
Q1 - My Age Is:
Under 22
22-25
26-35
36-45
46-55
56-65
Over 65
Q2 - I am a:
Male
Female
Q3 - I currently live in the following zip code ___________
Q4 - I currently receive the monthly CEA mailing:
Yes
No
Q5 - I prefer communication electronically (i.e. e-mail):
Yes
No
Q6 - I currently receive e-mail from the CEA:
Yes
No
11 | P a g e
Q7 - I donate to the CEA:
Yes (If yes, continue to question 8)
No (If no, continue to question 9)
Q8 – The total amount I donate to the CEA in a year is:
Less than $50
$50 - $100
$101 - $500
$501 - $1000
More than $5000 __________
Q9 – The reason why I haven’t donated to the CEA yet is:
Lack (or poor) communication with association
Don’t have the extra money
Donate to other organizations
I Don’t Donate
Q10 – When was the last time you visited the CEA website:
Today
Within the last week
Within the last month
Within the last year
I have never visited this website
Q11 – What would be the best way for you to receive information about the CEA:
U.S. Mail
Social Media
Magazine
I prefer not to receive any information about the CEA
Q12 – Are alumni relations important to you?
Yes, extremely important
Somewhat important
Slightly important
Not important at all
We have an alumni association?
12 | P a g e
Communication with the College of Engineering and Architecture
In this section, please circle the appropriate response to each question:
Q13 - The alumni association has great communication with its members.
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q14 – I would like to know how the alumni’s donations are being used by the CEA
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q15 – I visit the CEA website often
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q15 – When I see a piece of mail from the CEA, I always open it
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q16 – When I see an email from the CEA, I always open it
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q17 – Staying in contact with the CEA is important to me
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q18 – I follow the college on their social media sites
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q19 – I am interested in non-paper communication
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q20 – I like to keep up on what the CEA is doing
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
13 | P a g e
Q21 – Being a member of the CEA alumni is important to me
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q22 – I would volunteer for CEA alumni sponsored events if approached to do so
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q23 – I read anything that pertains to the CEA
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q24 – I realize that donations are important to the CEA
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q25 – Being able to have two-way communication with the CEA would make me consider
donating to the association
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
14 | P a g e
I presented our cover letter and survey to ten people between the ages of 22-70. It was
decided that five would be female respondents and five would be male respondents. All have
graduated from college with at least a Bachelor’s degree, and they were instructed to treat this
cover letter and survey as though they would have received it from their very own school. All
reported that the cover letter and survey were easy to understand and to navigate. The survey
averaged between 6-8 minutes to complete.
The feedback from the survey was interesting, as all have experienced their alumni
associations’ communication at one time or another. Two of the ten people stated that they
actively give to their alumni association. Four others indicated that they have given to the
association at one time or another. The remaining four have never given to the alumni
association for their school. They were able to understand the pretense of what the survey was all
about, merely by the scope of the questions asked. When asked if there were any questions that
they could think of that should have been included in the survey, they could think of none. They
agreed that the cover letter was friendly and straight to the point, which they appreciated since as
one participant put it, “these types of letters usually are full of too much information.”
As far as the survey questions were arranged, I had more questions to ask in the
demographic section, such as annual income and ethnicity, but I felt these questions would make
some respondents a little tentative about answering.
All in all, I feel that this pretest covers everything the association would be looking for.
From the answers given, they can form opinions on how to better communicate with the alumni,
15 | P a g e
which in turn will help them reach their goal of receiving donations from 35 percent or more of
the alumni.
16 | P a g e
Compared to the focus group, this method will be high in both reliability and accuracy.
Using the stratified random sampling method to select the participants of the survey is an
excellent method to produce results with high projectability; and as learned earlier – makes them
a good source of interpretations upon which to base future campaigns. I used the following
calculations to produce a sample size of 600 respondents, which will produce a 95% confidence
level with a +/- 4% error. To get to this number, we will send out surveys to 750 potential
participants.
n = (cl/ci)^2 (v)(1-v)
n = number of participants needed
confidence level (cl) = 1.96 sd
confidence interval (ci) = .04
variance (v) = .5
n= (1.96/.04)^2 (.5)(1-.5)
n= (2401)(.5)(.5)
n= 600
We will send out surveys to 750 CEA alumni members, knowing that the response rate
will be dictated by those who are non-responsive. We assume that those who are non-responsive
are probably some of the same people who do not read any of the CEA promotional materials.
We are predicting that we will end up with 600 CEA respondents that will answer the questions
and give us feedback with high validity that will benefit the CEA and its goals.
17 | P a g e
The Design
What we are trying to determine with this experiment is whether a group that is exposed
to more information about an organization, along with two-way conversation with staff, will
increase the possibility of the person donating to the association, compared with the person who
is shown general information only. We are working with independent variables here of increased
information about the association, as well as the two-way conversation with staff. The dependent
variable is the person’s likeliness to donate. The results of the experiment will be reported to the
department so that they can develop a communication plan that will encourage donations to the
program.
The sample will come from the CEA’s own list of alumni using a stratified random
sampling method. As with the focus groups, these experiments will take place in hotel meeting
rooms is Seattle, Spokane, and Vancouver. After all participants watch a video which shares
basic information on what the association is all about, they will take a 15 minute break and when
they come back, they will be split into two groups, one a control group and the other a test group.
There will be 200 respondents in each location. The experiment will be run under supervision of
CEA staff. The experiment will be best served under a controlled environment.
This experiment will help determine whether the survey could be expanded into other
sections of Washington State and other sections of the country.
18 | P a g e
How the Experiment will work
As stated earlier, participants will be in two groups of 100, with one a control group and
the other a test group. When the experiment begins, all participants (200) will sit in on a video
presentation, describing general ideas about what the association is, events it sponsors, number
of alumni, where the association is located, etc. They will then be a survey about their thoughts
on donating to the association. After a 15 minute break, the participants will be split into two
groups of 100. When the experiment continues, the control group will continue to see the video
of general CEA information, the test group will be joined by officers from the board of the
association and allowed to ask any questions they want concerning the association. They will
also see a video, but it will contain much more detailed information about the association. All
participants will then be asked a series of questions about donating to the association. The results
will indicate whether the “personalized” attention will be effective in promoting donations from
the participants.
The Survey
First I want to thank you all for participating in this survey. Your feedback is vital in improving
communications between the CEA and you – its alumni. There will be two parts of this survey –
Part 1 and Part 2. Please answer the questions after each session to the best of your ability.
Part 1
Q1 – I graduated from the CEA in the”
Pre 1950’s
1950’s
1960’s
1970’s
1980’s
19 | P a g e
1990’s
After 2000
Q2 – I found the information in the videos to be useful in teaching me what actually happens
with CEA
Yes
No
Q3 – I donate to CEA:
Monthly
Annually
In a Lump Sum
I have never donated to the CEA
Q4 – After watching this presentation, I know more about the CEA now. I would love to learn
even more.
Yes, I’m interested
No thanks
Q5 – The things that interested me the most in the presentation was (check all that apply):
CEA Activities
Chances to Volunteer
Research
How to donate
CEA Website
CEA Social Media Sites
Other _______________
Q6 – The CEA contributed to where I am today with my career
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q7 – I am all for helping current and future CEA students to the best of my ability
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
20 | P a g e
Q8 – I want to help CEA. This coming year I will donate
Less than $50
$50 - $100
$101 - $500
$501 - $1000
More than $5000
Q9 – When donating to a cause, I usually donate
Monthly
Annually
In a Lump Sum
I have never donated to a cause
Q10 – I understand that my financial support could have a great impact on the future of the CEA
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q11 – I want the CEA to stay in contact with me
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Q12 – It is really important to me that the CEA thrives and is successful
Strongly agree 1 2 3 4 5 6 7 8 9 10 Strongly disagree
Part 2
Welcome back – I hope you enjoyed your break. Please sit down and get ready for part 2 of our
session.
After participants are separated into both groups. The control group is shown more general
material about the CEA, while the test group is visited by association board members who take
their questions. When time is up – the participants of both groups are given the same survey as
they completed after part 1.
We want to thank you for the time you have given us today. Your feedback is very important and
it will be used to improve communications between the CEA and its alumni.
21 | P a g e
I like the idea that the experiment should be rationally valid in both internal and external
validity, but the strength of external validity will probably not be nearly as strong as the
quantitative survey taken earlier. Also, can we be sure that the answers about donating from the
participants are completely valid. Even though they know that their participation and their
responses will be private, there is a chance that they will think they are on the hook for donating
if they say that they will or might in the future. The amount they say they donate, both to the
program and by donating outside the program, may be overinflated by social pressures.
Also, are we to believe that the respondents that live in Seattle, Spokane, and Vancouver
are a true representation of alumni in places such as Walla Walla or Longview? We can’t answer
that at this time. However, we do know that the experiment will give the promotion, marketing
and public relations departments of CEA something more to work with.
22 | P a g e
The CEA wanted a consultant to come in and study the ways that that the public relations
department of Western State University could more effectively communicate with 20,000 CEA
alumni. They also wanted the consultant to help discover ways to increase the percentage of
alumni who give back to the college to at least 35 percent. By directing questions to the
participants of the focus groups, quantitative survey, and experiment groups, we were able to
establish the idea that better communication would mean better donations. By instituting two-
way conversation with the alumni, as well as increasing college and alumni ties through stronger
communication, the college should be able to reach their goal of receiving donations of 35% or
more in the future.