+ All Categories
Home > Documents > Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan&...

Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan&...

Date post: 12-Jan-2016
Category:
Upload: caitlin-poole
View: 218 times
Download: 0 times
Share this document with a friend
Popular Tags:
23
Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport
Transcript
Page 1: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Developing a Comprehensive Learning Outcomes Assessment Strategy for

Adult Learners

Cindy MacMillan& Susan Davenport

Page 2: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

2

Agenda

• Introductions

• Characteristics of Thomas Edison State College

• Challenges in an adult learning environment

Page 3: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

3

Introductions

• Susan Davenport, Dean of Heavin School of Arts and Sciences

• Cynthia MacMillan, Director of Outcomes Assessment and Accreditation

• Audience– Who’s here?– Where you are from?– Why this session?

Page 4: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

4

Why Learning Assessment?

• Is it really all about accountability?

• Accreditors• Students• Mission• You

Page 5: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

5

What are the challenges assessing learning in an adult environment?

• Heterogeneous population with different needs and skills

• Significant learning that the student brings to the learning environment

• Access to students for assessment

Page 6: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

6

Where to Start• Phase One – Define Learning Outcomes at Institutional and

Programmatic Levels

• Phase Two-- Operationalize the learning outcomes

• Phase Three – Implementation of assessment (finalize assessment matrix)

• Phase Four – Data collection and Analysis

• Phase Five -- Recommendations and transformations made in response to Phase Four data

Page 7: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

7

Where to start! (cont.)

Page 8: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

8

Phase OneDefine Learning Outcomes at Institutional and

Programmatic Levels

• Buy In• Strength in diversity• Define and Describe Goals and Process

– Institutional, Programmatic, Course

Page 9: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

9

Thomas Edison State College Institutional Outcomes

1. Information LiteracyThe Information literate graduate will demonstrate the ability required to recognize when information is needed and has the ability to identify, locate, evaluate and use effectively and appropriately the information needed.

2. Technological CompetencyGraduates will be able to solve problems and perform tasks using appropriate technological tools.

3. Responsible Ethical LeadershipGraduates will be able to articulate in their own words the ethical principles they have discovered and encountered in their academic discipline or profession. Graduates will recognize and articulate actual ethical dilemmas witnessed or experienced.

4. Scientific Reasoning Thomas Edison State College graduates will be able to analyze issues facing citizens by distinguishing scientific approaches from other ways to frame problems and issues.

5. Diversity/Global LiteracyThe Thomas Edison State College graduate will demonstrate an understanding of globally diverse perspectives, cultures, values, events, trends and issues.

Page 10: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

10

Thomas Edison State College Institutional Outcomes (contd.)

6. Quantitative Reasoning/LiteracyThomas Edison State College graduates will be quantitatively literate. They will be able to understand and apply mathematical concepts and skills to inform personal and professional decision-making.

7. Written and Oral CommunicationThomas Edison State College Graduates will be able to write and speak with proficiency.

8. Critical Analysis & ReasoningThomas Edison State College graduates will be able to demonstrate critical thinking and problem solving skills and are able to analyze content, discover meaning or significance, draw conclusions, and make assessments.

9. Life Long LearningGraduates will demonstrate a commitment to lifelong learning.

Page 11: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

11

Phase TwoOperationalize the Learning Outcomes

• Indirect versus direct measures

• Quantitative versus qualitative measures

• Triangulation

• Low versus High Stakes Assessment

• Baselines and Benchmarks

Page 12: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

12

Phase Two Continued

Methods we are using – • Standardized Assessment – ETS Proficiency

Profile, SAILS• Graduate Survey, Adult Learner Inventory• Capstones• Embedded Analysis• Historical Data Review• Survey• ETS Major Field Test• E-Portfolios

Page 13: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

13

Example: Assessment Matrix for Program Level Learning Outcomes

Bachelor of Science in Human Services outcomes:

Program Element which addresses this outcome

Measures (indicate direct or indirect) Benchmark/Target(acceptable and ideal if appropriate)

Interpret and critically analyze the research in the professional track

Course in Professional track

final project/paper A 70% earn a grade of B+I 80% earn a grade of B+ 

SOS 492 Research Methods

final research paper A 70% earn grade of B+ I 80% earn grade of B+ 

  ETS Proficiency Profile

Std. Test Critical Thinking A 60% achieve proficient or marginally proficientI 80% achieve proficient or marginally proficient

Apply theory to professional practice Capstone final paper A 70% earn grade of B+I 80% earn grade of B+

Apply knowledge of the specific skills, techniques, and agencies necessary to serve client populations

PSY 331 Intro Counseling

final exam A 70% earn grade of B+I 80% earn grade of B+

Capstone final paper A 70% earn grade of B+I 80% earn grade of B+

  Graduate Survey Item 7-a A 80% respond positivelyI 90% respond positively

Apply knowledge of cultural diversity as it relates to the field of Human Services. 

SOC 322 Cultural Diversity in the US

final exam A 70% earn grade of B+I 80% earn grade of B+

Capstone final paper A 70% earn grade of B+I 80% earn grade of B+

Apply theories of management as it relates to Human Services

MAN 301 Principles of Management

final exam A 70% earn grade of B+I 80% earn grade of B+

Capstone final paper A 70% earn grade of B+I 80% earn grade of B+

Page 14: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

14

Phase Three

Implementation of Assessment

• Be realistic in scope• Not everything needs to be assessed each year• Be pragmatic in tying plan to funds• Benchmarking

– Internal– External

• Rubric elements•

Page 15: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Information Literacy Data Collection Sample

15

Page 16: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Example: Data Collection for Human Services Degree

16

Bachelor of Science in Human Services outcomes:

Benchmark/Target

2010 results

Interpret and critically analyze the research in the professional track

70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant 

Above target

Apply theory to professional practice 70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant 

Below target

Apply knowledge of the specific skills, techniques, and agencies necessary to serve client populations

70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant 

Below target

Apply knowledge of cultural diversity as it relates to the field of Human Services. 

70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant 

Below target

Apply theories of management as it relates to Human Services

70% earn a score of 3 or above on a 4 point scale on rubric by reviewed independent consultant 

Above target

Page 17: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Phase Four and Five– Analysis and recommendations

17

Page 18: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Information Literacy - SAILS Recommendations

• Continue to evaluate student learning outcomes in the area of Information Literacy using the SAILS.

• Embedded SAILS in an upper level course for general populations.

• Identify and implement additional methods to assess the outcome area not assessed by SAILS.

• Include the SAILS in a few more courses to ensure that all degree programs of the College are included in this informational literacy assessment.

• Target the SAILS administrations to 250-275 students per year in order to facilitate longitudinal analysis in the future.

18

Page 19: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Information Literacy – ENC-102 Recommendations

• Expand the rubric elements to include one on grammar and spelling.

• Expand the data collection efforts in ENC-102 to include one section per semester.

• Expand the roll of the lead mentor to include collecting reliability data.

• Expand the use of Waypoint in other schools, courses and assignments.

• Explore viable options to communicate details of Waypoint use with mentors.

• Explore ways to minimize the use of instructional design resources.

• Seek efficiencies such as sharing rubric elements across schools.

19

Page 20: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

20

Human Services – Analysis and recommendations

Analysis of Human Services results led to the following curricular changes:

1. Modify the program description to more accurately describe the program and its objectives.

2. Modify the program outcomes to align with objectives in program by including diversity and theories of management.

3. Modify the general education requirements to 60 credits to align with the other degree offerings within the College. Require a three credit Ethics course under Humanities.

Page 21: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

21

Phase Four and Five– Analysis and recommendations (contd.)

4. Modify the Core Requirements. Based on guidelines from national accrediting bodies (i.e. The National Human Services Organization and the Council for Standards in Human Services Education), the core requirements were replaced with the following areas: Theoretical Foundation, Intervention, Client Populations, Research and Mgmt. This improves the quality of student learning outcomes and ensures that the program objectives are appropriate and measurable.

5. Modify the Practicum to a capstone course and objectives to make the practicum an effective instrument for assessing the degree outcomes.

6. Change the grading of the capstone from Credit/No Credit to a letter

grade. This will ensure a more accurate assessment of the outcomes of the course. A benchmark of “B+” was recommended.

Page 22: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

22

Your best practices, lesson learned and questions

Please share your feedback . . .

Page 23: Developing a Comprehensive Learning Outcomes Assessment Strategy for Adult Learners Cindy MacMillan& Susan Davenport.

Thank you for your time and interest!


Recommended