Date post: | 18-Dec-2014 |
Category: |
Documents |
Upload: | felicitymorgan |
View: | 551 times |
Download: | 0 times |
HOW DO YOU KNOW IT WORKS?
Measuring the Effectiveness of Career Counselling
Interventions
Joan McCurdy Myers
Felicity Morgan
HOW DO YOU KNOW IT WORKS?• UTMississauga Career Centre • Career Counselling Interventions and
Evaluation• Ace Your Career Case: content and
evaluation• Results• Challenges of integrating practice and
research• Next steps
UTMississauga CAREER CENTRE
• 10,500 students, mainly undergraduate• Broad range of academic programs• Large proportion of commuter students, living at home, GTA• Services include career counselling, employment advising,
workshops, panels, employer events, job postings• Staffing: 15 non-student staff
– 7(5.4 fte) staff in Career Development Unit
– 4 Employer Relations & Marketing
– 3 admin– 18 student staff
• Partnerships with academic programmes & student groups: key to tailoring for student needs
CAREER COUNSELLING INTERVENTIONS
• Individual, workshops & events• Some in class, some not• Workshops & events:
– Evaluate using post session evaluations, learning objectives, follow up surveys
– Students report high levels of satisfaction with sessions and meeting learning objectives
• Standard: 75% rate 4/5 on 5 point scale
EVALUATION OF INTERVENTIONS
• In 2006, decided to go beyond post-session satisfaction/feedback & surveys & attempt to measure what the students were truly learning– Our own questions– Learning outcomes focus at UT– Student Research Assistant position– Funding: through the Enhancing the Student
Experience Fund for two projects; Ace Your CareerCase and Intro Psych student research credit for Career Development workshop
ACE YOUR CAREER CASE
• 2nd and 3rd year Management and Commerce students
• Goal: career goal clarity: through self assessment, gathering of information and interaction with industry professionals
• Held annually in January for three years; facilitated by career counsellor and job coach consultant
ACE YOUR CAREER CASE
• 2 half days; Day One - self assessment through career
counselling exercises; career research methods and sources
Day Two – business introduction development and practice; lunch and interaction with employers; reflection on what was learned and next steps
• Co-sponsored with Management Department, funded by Provost’s Office
LEARNING OUTCOMES• Identify clear career goal, based on self
knowledge and industry knowledge• Locate and use career research
resources• Develop, practice and deliver business
introduction• Understand benefits and opportunities
available in employer events• Participate more effectively in recruitment
METHODS OF EVALUATION
• Post workshop evaluation• Follow up survey• Video-taped business introduction (years 2
& 3)• Students completed Career Decision
Profile pre Day One and post Day Two
CAREER DECISION PROFILE
• Developed by Prof. L. Jones (emeritus), North Carolina State, President Career Key Inc.
• Measures: – Decidedness– Comfort– Self Clarity– Knowledge about Occupations & Training– Decisiveness– Career Choice Importance
WHY THE CDP?
• Previously used it with at risk students; gave them clarity and purpose
• Scales break down the components of career planning, separates them out
• Questions are process oriented, not final answer oriented; fits with our approach
• Can be used as a counselling tool• Had support from the author
CDP SCALES
• Decidedness: how decided are you about your choice of career?
• Comfort: how comfortable are you with where you are in the process of making this choice?
• Self Clarity: how clear is your understanding of your abilities, interests and personality and how they fit with careers
SELF CLARITY SAMPLE QUESTIONS
• I wish I knew which occupations best fit my personality
• I need to have a clearer idea of what my interests are
• I need to have a clearer idea of my abilities, my major strengths and weaknesses.
• 8 point scale; strongly disagree to strongly agree
CDP SCALES
• Knowledge: how well informed you believe you are about occupations and educational programs that fit your interests
• Decisiveness: how capable you believe you are in making decisions without unnecessary delay
• Career Choice Importance: importance of choosing a career at this time
HOW EFFECTIVE WAS ACE?
• Results from post workshop evals• Results from follow up survey• Results from video-taped business intro• Results from CDP
RESULTS
• Post-workshop Evaluation– 100% recommended to a friend every year– Met learning outcomes criteria re: clarity of career
goal, strengthened career plan, more effective participation in recruitment,
– Did not meet learning outcomes criteria for: • The strength of their self marketing strategies • Satisfaction (wanted more time with industry professionals)
– Sometimes met: resources and how to use opportunities
RESULTS
• Follow Up Survey (one month post)– Overall, students felt all measures increased
significantly from pre-ACE to post-ACE• they understood themselves and the work world
better• they were more prepared for the recruitment
season• their career plan was more thorough and clear• their ability to self market had improved
RESULTS
• Video-taped Business Intro– Yr 2- 45% scored in 7-10 range, average 6.35
– Yr 3-66% scored in 7-10 range, average 7.4– Identifying 2 key strengths was a challenge
RESULTS• CDP
– Overall, scores moved in positive directions• Decidedness yielded mixed results
– Students significantly (<.05) increased their scores in knowledge 2 of 3 years (year 1 <.10)
– Year 3 saw the most significant changes: comfort, self clarity, knowledge
• Decidedness (significant but decreased)• Decisiveness (approaching significant <.10)
– Importance of Career Decision
RESULTS-for our student programming
• Great opportunity to experiment with new teaching/learning approaches– Business introduction (improved our teaching; used same
assessment in subsequent sessions)– Positive response of industry reps and students to reps
encouraged us to incorporate format in more of our activities
– Sharpened our focus on how to help students learn to reflect-incorporated direct reflection opportunities in a range of programming
– Style of greater student engagement on the spot also now used in Fall Management Conference with very positive results
CONCLUSIONS: How do you know it works?
• Take multiple perspectives to see what works– Students’ perceptions, satisfaction, learning
outcomes, feedback– “objective” measures e.g. CDP, behavioural
measurement e.g. business intros, 3rd party evaluations, rubrics
• Student participation in services, recruitment activities• Student “success” e.g. # interviews at recruitment
– Develop database for comparison e.g. UTM CDP
• Use results to improve programming
INTEGRATING PRACTICE & RESEARCH
• Proper evaluation without too much evaluation
• Need more time & expertise to plan, analyze, understand & use results
• Difficulty of balancing service provision with need for more structured assessment
• Speaking the language of the academic e.g. statistical results
INTEGRATING PRACTICE & RESEARCH
• Managing partnerships with academics is challenging
• Gratifying to be able to document & report changes
• Increased our skills and knowledge as professionals; identified new learning goals
• Benefits other students through improved programming
NEXT STEPS• Using CDP in other interventions; assists us to
assess where our students are at the beginning and improve our interventions to increase positive outcomes
• Follow up with participants• Encouraged us to go beyond satisfaction
measures; now “see” more opportunities for broader range of assessment/evaluation
• Increasing our expertise as a Centre & division• Looking for more partnerships