The Learning Outcomes Leaders Meeting with Deans

“Improvement—if it is to happen—will occur where faculty and staff have the most leverage to change how they approach teaching and learning.”

-Ewell, Paulson, and Kinzie, 2011 Feb. 6, 2015

Laura Blasi, Ph.D., Institutional Assessment

Meeting Goals…

1.Examine prior successes and challenges looking at the data we have from a college-wide perspective;

2.Review the Learning Outcomes Leader role and Work Plan sent with the invite for this;

3.Collaborate on a quick needs assessment related to our learning outcomes work at the college;

4.Analyze and make decisions – LOLs and deans together as possible – using prior reported data; and

5.Discuss spring assessments and begin to develop a plan for Assessment Day.

Most Students Experience Us as a College(not a program)

• Presentation to the Board of Trustees - 2014

Accomplishments, Challenges, Opportunities

• Strategic Goals • Essential Competencies• Faculty development• Annual review / workload• Curriculum Committee / catalog, • Viability and Academic Program Review (AS)

Assessment Day – 5 years of growth

– May 2010 – 11 College-wide Programs / Disciples participated (11% plans)

– May 2011 – 17 College-wide Programs / Disciplines Participated (25% plans)

– May 2012 – 61 Programs / Disciplines Participated (56% plans)

– May 2013 – 68 Programs / Disciplines Participated (100% plans)

– May 2013 – 64 Programs / Disciplines Participated (100% plans)

• IE Compensation Plan – a five-year cycle for assessing every outcome• Courses mapped to program / discipline student learning outcomes

2003 - The SACS Visiting Committee recommended that the College provide evidence as it relates to the quality of student learning outcomes.

11 college-wide assessment plans: May 2010

17 additional assessment plans: May 2011

56 college-wide assessment plans were implemented and improvement plans were developed - May 2012

However, the process was not yet uniform noted in a SACS recommendation following a visit in February 2013.

2014 visit and 2015 decision by SACS we were responsive

Important LessonCurricular Mapping is the Key to Moving From Course Level Assessment to Program Level Assessment

Course-Level Assessment

• Assess student learning outcomes at the end of the course

• Assign a grade to individual students

• Grading often involves only one faculty member who is teaching the course

Program-Level Assessment

• Assess student learning outcomes at the end of the program

• Evaluate aggregate student artifacts for purposes of program improvement

• Evaluation involves faculty teams across the program/ discipline

Note: The counts represent programs – the leader for each program took the survey.

Assessment Day 2014

Results / FindingsDocumenting the Impact

• Because faculty work together and share resources and are able to critique each other's resources students have better assimilated basic interpreting skills needed to be successful as students transferring as a junior to a university. Students also demonstrate these skills in various internship opportunities which they are more eager to apply.

• The group we assessed this year regarding demonstrating information literacy showed mixed levels of competency based on their performance on the research project. The weaknesses we identified were: the ability to write a good research question; and the ability to apply the information they found back to the research question.

Making Improvements

• Adjustments in our teaching and curriculum. Students seem to do well with the skills that are being taught but struggle with application and thinking/reasoning. We would like to provide more learning opportunities in the classroom to support mastery of these higher level thinking skills. Many faculty already do this, but we have discovered that many also do not. The data collected support this as we look at the large number of students that scored 1s on the competencies as they relate to thinking and reasoning skills but earned As or Bs in the course.

• We have some evidence to show that by implementing a “common tutorial”, we should be able to increase student learning as documented by our assessment. Faculty within the science discipline areas are developing tutorials using a tutorial template and a scientific article relative to the discipline area. The tutorial to be based on the discretion of the discipline task force and faculty.

• Improvement Plan: Projected Plans for Upcoming Year- • We determined that further additions of strategies are needed such as further explanation of problem instructions. • Determination of changes in group work assignment. (Some activities may work best when students complete as part of a team.) • Assessment at midterm of semester is recommended to determine those students that may need academic intervention.

Documenting the work with each outcome being assessed at least once in a five-year cycle

DiscussionProgram and DisciplineLearning Outcomes AssessmentSuccesses, Challenges, Resources

• Three or more successes

• Three or more challenges

• Three or more resources

Specific to assessment you have been part

of at the program or discipline level

Deans’ Perspectives – AS / Gen EdHow we support LOLs / LOA

Faculty Members’ Perspectives – AS / Gen Ed

What was learned from LOA

Learning Outcomes Leaders – Work Plan

Question shared in pre-survey…. “How do college leaders/administrates understand this role and what is expected from the Learning Outcome Leader?”

Advice from Experienced Learning Outcomes Leaders

• Include all faculty, full-time and part-time in the discussions and Assessment Day. Create clear guidelines for the assessment. Have an activity-driven agenda for Assessment Day so that actionable decisions can be made based on the data.

• Have meetings other than on Assessment Day to inform the faculty

about the progress and steps being made; many faculty hear about this work once a year and forget about it. Create smaller committees where faculty (other than the PLOA leaders) can work on different aspects of the PLOA work; this gets more faculty to engage.

• Offer clear communication to faculty in your program/discipline * Schedule meetings in advance to ensure participation * Partner with Dean to make this work a divisional effort; add to divisional meeting agendas.

x

X

Discussion

• Observations• Making it Meaningful• Next Steps

Discussion

• Spring Assessment Activities• Assessment Day• Long-term Planning

Challenges… momentum

Momentum, we have done so much good work and need to stay focused.

Finding or creating best practices in program assessment, and then educating faculty and staff in order to ensure the process is implemented efficiently and the data is accurate.

Ensuring that our program's learning program matches with what employers require from our students.

Challenges… meaningful assessment

In my opinion, the greatest challenge is to design a single assessment which will provide a meaningful results for all courses in a discipline.

Figuring out what data set to analyze. The college is so big, there are lots of avenues that could be pursued. I think it is difficult determining what type of analysis will yield the greatest benefit.

I think the system that is in place is starting to come together so I think at this point is it about the right artifacts, the methodology at times.

Challenges… college-wide communication

Trying to engage in meaningful conversation related to results, especially when this requires a willingness to discuss curriculum. Aligning assessments among and between disciplines so that student learning can be seen in progress and growth could be measured.

Getting buy-in from everyone involved. Communication has been the biggest problem I see.

Ensuring maximum college-wide faculty participation in the process and not leaving program/discipline learning outcomes assessment become the burden of the leader and a few faculty.

From Assessment Day 2014 Post-SurveyMaking the Day a Success

• Again, it all comes down to what the deans do or don't do with regard to this work. If the deans take it seriously the faculty will too. I have said that consistently since we started doing this work and I'll probably say it again next year.

• More opportunities to work across / with the disciplines. We would like to have a college-wide information literacy summit or big meeting.

• Training may be needed for leaders, deans, and chairs to better understand how to guide the faculty. If the leaders don't feel connected to the work, then the faculty concerns/fears become are echoed by the leader diminishing the quality.

Needs Beyond Assessment Day• Time to examine our needs. Dissemination of

knowledge is not an issue; re-examining what we want our students to know is most important. Retooling is important at every junction.

• Stronger communication among faculty, faculty development opportunities related to specific assessments, and time spent developing clearly measurable criteria in the rubric.

• More people who understand the value and importance

of assessment involved in the process. I think many see it as something we have to do, but they do not understand the possible benefits, because I'm not sure how authentic the assessment has been and the results have not been broadly shared by everyone in the college.

Closing Conversation and Evaluation

• Results from meeting returned to you by Feb 20• Please make Organizer corrections / additions by March 20• Let us know by March 20 if you would like a Blackboard

shell to use to organize the work• Deans ask for an Assessment Day Update prior to April 10

• Valencia Faculty-Developed Assessment Glossary:• http://valenciacollege.edu/instassess/documents/lacGlossary.pdf