Post on 02-Sep-2020
transcript
Assessing the
Citizenship ISLO:
Spring 2014
FACILITATOR: KAREN WONG
WITH MUCH GRATITUDE TO THE PARTICIPATING INSTRUCTORS
SOURCE: SKYLINE COLLEGE OFFICE OF PLANNING, RESEARCH, AND INSTITUTIONAL EFFECTIVENESS
Citizenship ISLO 1) DEMONSTRATE SCIENTIFIC LITERACY CONCERNING A RANGE OF
GLOBAL ISSUES.
2) ARTICULATE SIMILARITIES AND CONTRASTS AMONG CULTURES,
DEMONSTRATING KNOWLEDGE OF AND SENSITIVITY TO VARIOUS
CULTURAL VALUES AND ISSUES.
3) DEVELOP ATTITUDES CENTRAL TO LIFELONG LEARNING: OPENNESS,FLEXIBILITY, INTELLECTUAL CURIOSITY, AND A BROAD PERSPECTIVE THAT
VALUES DIVERSITY OF THOUGHT.
4) DEMONSTRATE APPROPRIATE SOCIAL SKILLS IN GROUP SETTINGS,
LISTENING AND BEING RECEPTIVE TO OTHERS’ IDEAS AND FEELINGS,
EFFECTIVELY CONTRIBUTING IDEAS, AND DEMONSTRATING LEADERSHIP
BY MOTIVATING OTHERS.
5) DEMONSTRATE COMMITMENT TO ACTIVE CITIZENSHIP.
Assessment Strategy
• Eighteen faculty across the disciplines whose
courses mapped to at least two of the five
descriptors participated.
• Faculty evaluated 523 students’ work– a major
assignment, performance, or activity-- using the
relevant parts of the rubric.
• The PRIE office aggregated and disaggregated
results for discussion.
Citizenship Rubric: Novice, Basic, Proficient, Advanced
• Scientific Literacy
• Scientific Method
• Understanding of the impact of decisions made by local, national and international
organizations, societies, environments and economies
• Understanding of the impact of the global economy on political decision-making
• Understanding of the interconnectedness of the world
• Participation in a global society
• Cultural Sensitivity
• Lifelong Learning
• Social Skills
• Social Interaction
• Participation
• Teamwork
• Leadership
• Active Citizenship
Rubric Example: Demonstrate
commitment to active citizenship.
Indicator □ Novice □ Basic □ Proficient □ Advanced
Demonstrate commitment to active citizenship via civic engagement: having a sense of responsibility to the larger community
No commitment- student may be aware of issues but does not participate in active citizenship.
Minimal commitment-student is aware of issues; may make conscious choices and change own daily habits.
Adequate commitment-student demonstrates “basic” skill level behaviors; in addition, informs and attempts to persuade family and friends to make similar changes and/or collaborates with others to enact change.
Full commitment-student demonstrates “proficient” skill level behaviors; in addition, adopts a leadership role in enacting change.
EIGHTEEN PARTICIPATING
DISCIPLINE FACULTY
Communications (3)
English (3)
English for Speakers of Other Languages (1)
World Languages (1)
Kinesiology– Varsity
Teams (2)
Administration of Justice (1)
History (1)
Paralegal Studies (1)
Political Science (1)
Biology (2)
Physics (1)
Surgical Technology (1)
Telecommunications (1)
SAMPLE SIZE
REPRESENTATION
3%
53%22%
22%
Kinesiology Language Arts Social Sciences Science/ Technology
Assignments/ Activities Activity
Students worked with a partner and then reflected on what they learned about their partner’s culture, and which intercultural communication competency skills might help them/did help them through this process.
Major Writing Assignment
Students researched and wrote about ecological issues; the psychological, political, social and economic contexts of crime; the causes of entrenched poverty in a third world country, etc.
Students reflected on how their personal actions and food consumption affect the world around them.
Performance and Reflection
Students who play team sports; participate in science labs; are on site for clinical observations were evaluated and/or evaluated each other’s social skills.
Exams
Students answered multiple choice test questions on their legal rights and the courts, and indicated whether they knew the information prior to taking the course.
Guiding Questions
• WHICH OF THE DESCRIPTORS ARE MOST PREVALENT IN
THE COURSES THAT WERE ASSESSED?
• HOW ARE STUDENTS FARING WITH EACH?
• WHAT ARE THE IMPLICATIONS OF THESE FINDINGS?
Which of the rubric descriptors are most prevalent in the
courses that were assessed?
10783
63
154179
316
234
287 287303
284
192
0
50
100
150
200
250
300
350
Scientific Method (SM) SM- Impact of Decisions SM- Impact of Global Economy
SM- World Interconnectedness SM- Global Participation Cultural Competency
Lifelong Learning Social Skills (SS) - Interaction SS- Participation
SS- Teamwork SS- Leadership Active Citizenship
How are students faring?
Mastery Which criteria have the highest number of “advanced” and “proficient” scores?
Challenge(s) Which criteria have the highest number of “novice”scores?
Insights?
Concerns?
Questions?
20% 18%13%
17%11%
33%26% 28%
20% 18%24%
11%
36% 40%
35%
41%
36%
43%
43%46%
40%49%
47%
48%
35%37%
46%34%
45%
19%
20%18%
22%
27% 24%
34%
9%5% 6% 8% 7% 5%
10% 8%
19%
5% 6% 7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Overall Results
Advanced Proficient Basic Novice
How are students faring
with social skills?
3326 28
20
43
4346
40
1920
18
22
510 8
19
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Social Interaction Participation Teamwork Leadership
Advanced Proficient Basic Novice
Equity–
How are different groups faring?
What are the various levels of mastery for
each group?
Who, if anyone, appeared to be
disproportionately impacted?
What may account for different group’s
outcomes?
Disaggregated by Gender
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
Female 51 % 53 % + 2 %
Male 47 % 43 % - 4 %
Unreported 2 % 4 % + 2 %
Disaggregated by Gender
22%16%
27%
37%43%
41%
19%
31%
30%
22%
11%
2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Female (1434) Male (1151) Other/ Unreported (103)
Advanced Proficient Basic Novice
Disaggregated by Ethnicity
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
African American 3 % 3 %
Asian/ Pacific Islander 21 % 23 % +2 %
Filipino 17 % 18 % +1 %
Hispanic 18 % 18 %
White 19 % 12 % - 7 %
Multi-ethnic 18 % 16 % - 2 %
Unreported 3 % 5 % + 2 %
Disaggregated by Ethnicity
17%20%
11%
22%18%
23%
30%25%
67%
40%54%
45%
43%
42%
49%
38%
17%
27%
29% 26%
29%27%
21%
30%
0%
12%6% 7% 10% 8%
0%
7%
0%
20%
40%
60%
80%
100%
120%
American Indian
(6)
Asian/ Pacific
Islander (576)
African
American (82)
Filipino (462) Hispanic (469) Multi-ethnic (406) White (302) Other/
Unreported (124)
Advanced Proficient Basic Novice
Disaggregated by Age
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
Under 18 6 % 1 % - 5 %
18-22 53 % 67 % +14 %
23-28 23 % 18 % - 5 %
29-39 11% 8 % -3 %
40-49 4 % 3 % -1%
50-59 2 % 1% -1%
60+ 1% 3 % +2 %
Disaggregated by Age
31%
21%15%
27%
35%39%
32%
69%
40%49%
60% 46%
30% 43%
0%
29%28%
13%
11%
26%
24%
0%
10% 8%
0%
8%4%
1%
0%
20%
40%
60%
80%
100%
120%
Under 18 (13) 18-22 (1634) 23-28 (429) 29-39 (206) 40-49 (63) 50+ (23) Other/ Unreported
(79)
Advanced Proficient Basic Novice
Positive Findings (from 9/22/14 Town Hall)
• ASSESSED STUDENTS ARE SOCIALLY COMPETENT.
• FEMALES ARE MORE HIGHLY REPRESENTED ON OPPOSITE ENDS OF
THE SPECTRUM, “ADVANCED” AND “NOVICE.” ON THE OTHER
HAND, ALL THREE GENDER CATEGORIES (INCLUDING
“OTHER/UNREPORTED”) ARE ABOUT EVENLY SPLIT BETWEEN
“NOVICE AND BASIC” AND “PROFICIENT AND ADVANCED.”
• INSTEAD OF ONLY FOCUSING ON ASSESSMENT AS THE END
PRODUCT, WE’RE MOVING IN THE DIRECTION OF USING THE
SLOAC AS A MEANS TO HAVE MORE DISCUSSIONS AROUND
CURRICULUM AND TEACHING PRACTICES, THE “INPUT” THAT
ENABLES STUDENTS TO PRACTICE AND DEVELOP THESE
COMPETENCIES.
Areas for Additional Dialogue(from 9/22/14 Town Hall)
• HIGHEST PERCENTAGE OF “NOVICES” WAS UNDER
“LEADERSHIP.”
• THE RESULTS MAY BE DUE TO THE TYPES OF ASSESSMENTS AND/OR CONTEXT OF
LEADERSHIP BEING ASSESSED.
• LOWEST LEVELS OF “MASTERY,” AND HIGHEST LEVELS OF
“BASIC/NOVICE” ALL INVOLVE GLOBAL CRITERIA.
• PERHAPS GLOBAL TOPICS ARE NOT REINFORCED ENOUGH IN OUR
CURRICULUM.
• POSSIBLY THE AGE OF THE STUDENTS ALSO FACTORED INTO THE RESULT. THE
VAST MAJORITY ARE 18- 22 AND MAY NOT HAVE BEEN SUFFICIENTLY
EXPOSED TO GLOBAL TOPICS.
• ALSO THERE WERE LOW LEVELS OF REPRESENTATION OF THOSE AREAS IN THE
COURSES SAMPLED.
Potential Action Plans (from 9/22/14 Town Hall)
LEADERSHIP CRITERION
Professional development-- May want to initiate more conversation about how these ISLO competencies manifest themselves within our disciplines, and techniques for both teaching and assessing each ISLO component.
GLOBAL CRITERIA
May want to offer more professional development about how we can incorporate global topics into our curriculum.
May want to launch a “One [Global] Book” initiative such that students study a common book from different disciplinary perspectives.
May want to integrate more service learning opportunities into classes.
May want to disaggregate the data according to the number of units students have taken to see if there’s a difference between the two groups (e.g., 36+ units and those with fewer).
Is there a better way to select courses so that representation of the areas is better balanced? If more courses had this as a focus, the results on the rubric might be different.
May want to prompt people to review their mapping, which could help to improve the sampling of courses used for ISLO assessments.
Action Plan: What might you do in
response to these ISLO findings?
How is this information pertinent to your discipline/ area?
What can you do to reinforce or improve learning?
Revise how you convey the information?
Revise the assignment/ task?
Provide more opportunities for practice and feedback?
Revise assessment methodology?
Revise ISLO descriptors?