1
Additional Findings from Participation in
MSL 2015 - Elon UniversityApril 7, 2016
Steve Mencarini, Bob Frigo, and Kim Fath
MSL/ NCLP, 2006, 2009, 2010, 2011, 2012, 2015
Presentation Outline
• Purpose & Frameworks for the Study
• Methodology
• Highlights of Elon’s Findings:
– Leadership
– Service-Learning
– Additional Campus Analyses
2
Purpose of the MSL
To examine student leadership values at both the
institutional and national levels with specific attention
to the campus experience factors that influence
leadership development in college students.
3
To build an international
research program that
advances college student
leadership development.
Framework of the MSL
Theoretical Framework:
Social Change Model of Leadership
Development (HERI, 1996)
Conceptual Framework:
I-E-O College Impact Model
(Inputs-Environment-Outcomes)
(Astin, 1993, 2001)4
The Social Change Model
5
Social Change Model: 7 C’s
Individual Values:
- Consciousness of Self
- Commitment
- Congruence
Community Value:
- Citizenship
6
Group Values
- Collaboration
- Common Purpose
- Controversy with Civility
Methodology
The Survey Instrument:
• Web-based (link sent via e-mail) during
February 2015
• Average completion time of 24 minutes across
all schools
• Schools could also ask up to 10 institution-
specific custom questions
• Survey management by Survey Sciences Group
7
Methodology
Ninety-seven institutions participated in 2015
– Geographically diverse
– Variety of institutional types and classifications
e.g., community colleges, women’s colleges, research
universities, liberal arts schools, HBCU’s and HSI’s
Institutions such as W&M, Emory, Georgetown, UNC-CH,
Lehigh, NC State, Northwestern, etc.
– Differing levels of leadership programming
(extensive at some, nascent at others)
– See www.leadershipstudy.net for complete list8
Methodology
Random Samples:
• A random selection of undergraduate students from
each institution (4,000 for most) was invited to
participate; total population was used for campuses
with 4,000 or fewer students
– Elon participated with 4,000 randomly sampled students
Comparative Sample:
• Each institution was allowed to identify custom
populations to serve as comparison groups
– Participants in CFL’s LEAD program9
Methodology
Data Collection & Results
• Standard data cleaning techniques were employed
• Findings were generated using descriptive and
inferential statistical methods and summarized in
final report for each institution
• Raw data was provided to schools for additional
analyses
• Incentives including gift cards to restaurants and
Phoenix cash
10
11
Select
Findings
Response Rates
Our Institution 2015
National
Sample
Invited 4,000 311,678
Responded 1,664 96,588
Response rate 41.6% 31.0%
12
Demographics
Our Institution 2015 National Sample
Gender
Female 69.59% 64.58%
Male 30.41% 35.42%
Class
First-Year 23.9% 23.5%
Sophomore 19.2% 21.62%
Junior 22.9% 25.2%
Senior 34.0% 29.68%
13
Demographics
Our Institution 2015 National Sample
Race/ethnicity
White/Caucasian 89.2% 67.51%
Middle Eastern/Northern African - 0.71%
African American/Black 7.2% 5.31%
American Indian/Alaska Native - 0.29%
Asian American 3.4% 6.99%
Native Hawaiian/Pacific Islander - 0.3%
Latino/Hispanic 4.6% 6.28%
Multiracial - 10.01%
Race Not Listed - 2.59% 14
What are the overall
findings about leadership?
2015 National
Sample
General Elon
student
LEAD participants
Consciousness of Self 4.05 4.08 4.26*
Congruence 4.24 4.30* 4.43*
Commitment 4.40 4.47* 4.54
Collaboration 4.18 4.23* 4.35
Controversy with Civility 4.23 4.26 4.35
Citizenship 3.94 4.03* 4.37*
OMNIBUS SRLS (total
score)
4.17 4.23* 4.38*
15
* Indicates p<.01
Is there a difference in leadership
between the class years?
First Year Soph. Junior Senior
Consciousness of Self 4.014 4.004 4.094 4.231,2,3
Congruence 4.28 4.26 4.31 4.34
Commitment 4.44 4.41 4.5 4.51
Collaboration 4.184 4.154 4.27 4.341,2
Controversy with Civility 4.214 4.194 4.28 4.351,2
Citizenship 4.00 3.944 4.08 4.112
OMNIBUS SRLS (total score) 4.184 4.164 4.25 4.311,2
16
* Indicates p<.01
Is there a difference in leadership
between males and females?
Female Male
Consciousness of Self 4.09 4.06
Congruence 4.342 4.211
Commitment 4.512 4.381
Collaboration 4.262 4.161
Controversy with Civility 4.302 4.161
Citizenship 4.082 3.911
OMNIBUS SRLS (total score) 4.262 4.141
17
* Indicates p<.01
18
White/
Caucasian
African
American/
Black
Asian American Latino/Hispanic Multiracial
Consciousness of
Self4.09 4.21 3.91 4.03 4.03
Congruence 4.3 4.41 4.11 4.40 4.28
Commitment 4.47 4.56 4.31 4.48 4.47
Collaboration 4.24 4.30 4.13 4.07 4.18
Civility 4.25 4.33 4.09 4.28 4.28
Citizenship 4.02 4.19 3.95 4.07 4.06
OMNIBUS SRLS
(total score)4.23 4.34 4.08 4.21 4.21
* Indicates p<.01
Is there a difference in leadership
broken out by race?
Is there a difference in
leadership based on level of
faculty mentor relationship?
Never Once Sometimes Often
Consciousness of Self 3.903,4 3.904 4.051,4 4.221,2,3
Congruence 4.133,4 4.204 4.261,4 4.421,2,3
Commitment 4.303,4 4.304 4.421,4 4.611,2,3
Collaboration 4.063,4 4.094 4.201,4 4.361,2,3
Controversy with Civility 4.073,4 4.144 4.231,4 4.381,2,3
Citizenship 3.783,4 3.914 4.001,4 4.191,2,3
OMNIBUS SRLS
(total score)
4.043,4 4.094 4.191,4 4.361,2,3
19
* Indicates p<.01
Is there a difference in
leadership based on level of
staff mentor relationship?
Never Once Sometimes Often
Consciousness of Self 4.014 4.024 4.11 4.231,2
Congruence 4.264 4.224 4.304 4.431,2,3
Commitment 4.393,4 4.454 4.491,4 4.611,2,3
Collaboration 4.153,4 4.194 4.261,4 4.421,2,3
Controversy with Civility 4.193,4 4.204 4.301 4.381,2
Citizenship 3.913,4 4.004 4.081,4 4.261,2,3
OMNIBUS SRLS
(total score)
4.153,4 4.184 4.251,4 4.391,2,3
20
* Indicates p<.01
Study
Abroad
Internship Research Service
learning
Consciousness of
Self
4.16* 4.14* 4.14 4.15*
Congruence 4.35* 4.33 4.36 4.37*
Commitment 4.51* 4.50* 4.53* 4.54*
Collaboration 4.30* 4.30* 4.30* 4.33*
Controversy with
Civility
4.32* 4.30* 4.34* 4.33*
Citizenship 4.11* 4.10* 4.14* 4.21*
OMNIBUS SRLS
(total score)
4.29* 4.28* 4.30* 4.32*
21* Indicates p<.01; please note that the significant findings are comparing those
who did and did not participate in the experience, not between experiences
Is there a difference in leadership
based on participation in the 4
other Elon experiences?
Never Once Some-
times
Many
Times
Much of the
Time
Consciousness of Self 3.775 4.02 3.925 4.035 4.241,3,4
Congruence 4.055 4.28 4.185 4.245 4.431,3,4
Commitment 4.225 4.45 4.345 4.425 4.591,3,4
Collaboration 3.944,5 4.105 4.105 4.201,5 4.381,3,4
Controversy with Civility 4.065 4.055 4.185 4.205 4.381,2,3,4
Citizenship 3.503,4,5 3.704,5 3.901,5 4.001,2,5 4.221,2,3,4
OMNIBUS SRLS
(total score)
3.924,5 4.095 4.105 4.181,5 4.371,2,3,4
22
* Indicates p<.01
Is there a difference in leadership
based in how involved you are?
Never Once Sometimes Many
Times
Much of
the Time
Consciousness of Self 3.964,5 3.924,5 4.075 4.151,2,5 4.381,2,3,4
Congruence 4.245 4.164,5 4.265 4.352 4.491,2,3
Commitment 4.435 4.365 4.415 4.495 4.641,2,3,4
Collaboration 4.144,5 4.145 4.225 4.311 4.441,2,3
Controversy with Civility 4.205 4.145 4.235 4.31 4.421,2,3
Citizenship 3.894,5 3.824,5 4.015 4.121,2,5 4.391,2,3,4
OMNIBUS SRLS
(total score)
4.144,5 4.094,5 4.205 4.281,2,5 4.461,2,3,4
23
* Indicates p<.01
Is there a difference in leadership
based if you have a leadership role?
HS
participa-
tion
HS
leader-
ship role
College
participa-
tion
College
leader-
ship role
Leader-
ship
minor
Leader-
ship
workshop
HS part. 1
HS lead.
role
.521* 1
College
part.
.221* .203* 1
College
lead. role
.137* .204* .586* 1
Lead.
minor
.117* .137* .135* .167* 1
Lead.
workshop
.169* .169* .306* .234* .417* 1
24
What is the correlation between
various leadership involvements?
* Indicates p<.01
R- square B β
Step 1 .336*
Gender -.079 -.06
Academic level .039 .105
HS leadership
roles
.034 .083
Pre-SRLS score .445 .496
Step 2 .366*
College
leadership roles
.051 .189
Step 3 .370*
LEAD
participation
.104 .071
25
Does the LEAD Program have an
impact on leadership?
* Indicates p<.01
ADDITIONAL CAMPUS ANALYSES
26
Campus Rec employees
leadership capacities
• Compared employees to non-employees
• Significant results (p<.05): Overall socially
responsible leadership, congruence, collaboration,
and citizenship; sense of belonging; resilience;
leadership efficacy
• Non-significant results: consciousness of self,
commitment, controversy with civility (although
the means for the employees were higher in all
three categories, it was not significant)
27
Campus Rec:
Involvement in Programs
• Instructor-led group fitness: correlates to ALL leadership outcomes
(consciousness of self, congruence, commitment, collaboration,
controversy with civility, citizenship), leadership efficacy, sense of
belonging
• Intramural sports participation: none of the leadership outcomes,
however it does correlate with leadership efficacy, resilience, and
sense of belonging
• Open rec: consciousness of self, commitment, collaboration, overall
socially responsible leadership, leadership efficacy, resilience, and
sense of belonging
• Outdoor adventure: social change behaviors, collaboration, controversy
with civility, citizenship, leadership efficacy, resilience, and sense of
belonging
28
Kernodle Center:
Service Hours
• 2014-2015 Total Hours of Validated Service Hours: 128,869
MSL Service Hour Data Harvesting
• Questions: In an average month, do you engage in any community service?
(1=Yes 2=No)
• In an average month, approximately how many hours do you engage in
community service…
• As part of a class?
• As part of a work study experience?
• With a campus organization?
• As part of a community organization unaffiliated with your school?
• On your own?
MSL Service Hour Approximation: 174,341
29
Kernodle Center: Student
Impact Assessment Project
• Primary Goals
– Understand how service-learning impacts the knowledge, skills, attitudes,
and public action of students to identify needed improvements
– Determine if service-learning can take place in a co-curricular context in
addition to a credit-bearing context
• Research Questions
1.) How is the Kernodle Center preparing students to be engaged global citizens
and informed leaders motivated by concern for the common good?
2.) What knowledge do students gain from participating in service-learning?
3.) What skills do students gain from participating in service-learning?
4.) What attitudes do students develop from participating in service-learning?
5.) What public action experiences do students gain from participating in
service-learning?
30
Kernodle Center: Student
Impact Assessment Project
• Survey Models
– DukeEngage Group Programs Outcomes and Community Impacts
Evaluation
– Civic Minded Graduate Scale and Narrative Prompt
• Survey Organizing Frameworks
– Kernodle Center for Service Learning and Community Engagement
Student Learning Outcomes
– Civic Learning Spiral
• Proposed Assessment Instruments
– Pre-Experience Survey Instrument and Narrative Prompt
– Post-Experience Survey Instrument and Narrative Prompt
– Six Months Post-Experience Survey Instrument and Narrative
Prompt31
INSTITUTIONAL ASSESSMENT
32
Where we started
FIRST GENERATION GENDER RACE
RESIDENTIAL
STATUS
SEXUAL
ORIENTATION
Non 1st
Gen 1st Gen F M White
Nonwhit
e Res Comm Het. LGBQ
I feel valued as a person at
this school
I feel accepted as a part of the
campus community
I have observed discriminatory
words, behaviors or gestures
directed at people like me
I feel I belong on this campus
I have encountered
discrimination while attending
this institution
I feel there is a general
atmosphere of prejudice
among students
Faculty have discriminated
against people like me
Staff members have
discriminated against people
like me
Sense of Belonging
Climate
Descriptor Response Options
Indicate your level of agreement with the following statements about your
experience on your current campus:
I feel valued as a person at this school 1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
I feel accepted as a part of the campus community
I feel I belong on this campus
Non-Discriminatory
Climate: Indirect
Descriptor Response Options
Indicate your level of agreement with the following statements about your
experience on your current campus:
I have encountered discrimination while attending
this institution 1 = Strongly Agree
2 = Agree
3 = Neutral
4 = Disagree
5 = Strongly Disagree
I feel there is a general atmosphere of prejudice
I would describe the environment as
negative/hostile
Non-Discriminatory
Climate: Direct
Descriptor Response Options
Indicate your level of agreement with the following statements about your
experience on your current campus:
Faculty have discriminated against people like me 1 = Strongly Agree
2 = Agree
3 = Neutral
4 = Disagree
5 = Strongly Disagree
Staff members have discriminated against people
like me
Other students have discriminated against people
like me
Defining Social
Identities
I’m a
cisgender
female
I’m a
transgender
male
I’m Puerto-
Rican
I’m gay
I have an
anxiety
disorder
I have a
visual
processin
g
disorder
My father went
to college but
didn’t finish
My
mother is
Latino
and my
father is
White
I’m the first in
my family to go
to college
I’m
bisexual
First Generation
Neither parent/guardian has completed a
baccalaureate degree (Elon Definition)
**May have attended
college or earned
an associates degree
Gender Identity
Could not include students who did not
identify specifically as male or female
Race/Ethnic Identity
What do we use
to assign a
student to
ALANAM
membership?
Results by identity
Scale
Parent
Education Gender Race/Ethnicity
Sexual
Orientation Ability Status
Firs
t-G
en
No
n-
Firs
t Ge
n
Ma
le
Fe
ma
le
ALA
NA
M
No
n-A
LA
NA
M
LG
BQ
He
tero
se
xua
l
Ide
ntifie
d
Dis
ab
ility
No
ne
Sense of
Belonging3.50 3.83 3.78 3.80 3.51 3.85 3.49 3.81 3.53 3.83
Non-
discriminatory
Climate –
Indirect
3.20 3.60 3.48 3.59 3.14 3.64 3.07 3.59 3.33 3.59
Non-
discriminatory
Climate – Direct
3.80 4.24 4.06 4.25 3.77 4.28 3.74 4.22 3.99 4.22
[1] A student is classified as first-generation if neither parent has earned a baccalaureate degree (Elon definition)[2] Transgender students omitted from analysis due to insufficient number of respondents in that identity group.[3] Includes students who identify from at least one of the following groups: Middle Eastern/North African, African American/Black, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Asian American, Latino/Hispanic/Multiracial, or Race Not Listed
Are the differences
statistically significant?
Scale
Parent
Education Gender
Race/Ethni
city
Sexual
Orientation
Ability
Status
Sense of
BelongingX - X X X
Non-
discriminat
ory Climate
– Indirect
X X X X X
Non-
discriminat
ory Climate
– Direct
X X X X X
[1] A student is classified as first-generation if neither parent has earned a baccalaureate degree (Elon definition)[2] Transgender students omitted from analysis due to insufficient number of respondents in that identity group.[3] Includes students who identify from at least one of the following groups: Middle Eastern/North African, African American/Black, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Asian American, Latino/Hispanic/Multiracial, or Race Not Listed
How much of a
difference?
Scale
Parent
Education Gender
Race/Ethnic
ity
Sexual
Orientation
Ability
Status
Sense of
Belonging● - ● ● ●
Non-
discriminato
ry Climate –
Indirect
● ○ ● ● ○
Non-
discriminato
ry Climate –
Direct
● ○ ● ● ○
[1] A student is classified as first-generation if neither parent has earned a baccalaureate degree (Elon definition)[2] Transgender students omitted from analysis due to insufficient number of respondents in that identity group.[3] Includes students who identify from at least one of the following groups: Middle Eastern/North African, African American/Black, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Asian American, Latino/Hispanic/Multiracial, or Race Not Listed
How do our results in
2015 compare to 2012?
Sense of Belonging
2012 2015 Effect
Ability Status
Identified Disability 3.68 3.53 -
No Identified Disability 3.92 3.83 .12
Sexual Orientation
LGBQ Students 3.55 3.49 -
Non – LGBQ Students 3.92 3.81 .14
Race/Ethnicity
ALANAM Students 3.76 3.52 .26
Non- ALANAM Students 3.94 3.85 .10
Gender
Male 3.84 3.78 -
Female 3.94 3.80 .17
Parent Education
First Generation 3.73 3.50 .25
Non-First Generation 3.93 3.83 .12
[1] Includes students who identify from at least one of the following groups: Middle Eastern/North African, African American/Black, American Indian/Alaska Native,Native Hawaiian/Pacific Islander, Asian American, Latino/Hispanic/Multiracial, or Race Not Listed
Sense of Belonging
2012 2015 Effect
Ability Status
Identified Disability 3.68 3.53 -
No Identified Disability 3.92 3.83 .12
Sexual Orientation
LGBQ Students 3.55 3.49 -
Non – LGBQ Students 3.92 3.81 .14
Race/Ethnicity
ALANAM Students 3.76 3.52 .26
Non- ALANAM Students 3.94 3.85 .10
Gender
Male 3.84 3.78 -
Female 3.94 3.80 .17
Parent Education
First Generation 3.73 3.50 .25
Non-First Generation 3.93 3.83 .12[1] Includes students who identify from at least one of the following groups: Middle Eastern/North African, African American/Black, American Indian/Alaska Native,Native Hawaiian/Pacific Islander, Asian American, Latino/Hispanic/Multiracial, or Race Not Listed
Do students who were
here in 2012 feel
differently in 2015?
How do our results
compare to the national
data?
IS THE CLIMATE THAT CHILLY FOR
OUR STUDENTS?
IT DEPENDS ON YOUR QUESTION
50