Investigating the IMPACT OF LMIResearch TeamRéginald Savard, Sylvain Paquette, Céline Renald and Lynne Bezanson in collaboration with Bryan HiebertCanadian Career Development Foundation (CCDF) Canadian Research Working Group in Evidence-Based Practice (CRWG)
In partnership with New Brunswick Post-Secondary, Education and Labour
3
Review of the Problem•Most research on LMI focuses on usability of
products:▫Readability, accuracy of information▫Ease of access, amount of use▫Most research is with students: very little with adults▫Little literature about the direct contribution of LMI▫ Savard, R., Michaud, G., Bilodeau, C. et Arseneau, S. (2007). L’effet de l’information sur le marché du
travail sur le processus décisionnel relatif au choix de carrière. Revue Canadienne de Counseling, 41(3), 158-172.
•Several questions remain unanswered▫How do people use LMI?▫What (if any) assistance would be helpful?
4
Research Questions▫If client needs are assesed and clients are given
LMI consistent with their needs:
▫to what extent does assistance by a service provider enhance their effective use of LMI?
▫and
▫to what extent is independent self-help a sufficient process for clients to use LMI effectively?
5
Review of foundations for the study• The study will take place within current service
delivery practices in the employment centres ▫so that processes can be incorporated into daily
practice if research results prove positive.•Career practitioners and clients will work in their
customary settings• LMI interventions will be isolated▫LMI is a distinct intervention, separate from
employment counselling or worksearch workshops in which LMI may be a component
▫There will be separate LMI interventions for career decision making and for job search
6
Procedures followed for the francophone sample• Translation of LMI Booklets (client and resource
centre)▫Career Decision Making (2)▫Work Search (2)
• Field testing▫Duration: 4 weeks
•Semi-structured interviews with clients ▫1 week and 4 months after entering the research
•Semi-structured interviews with research partners (career practitioners, managers) ▫several weeks after the end of the research
7
Initial Contact: Employability Assessesment Interview
• Career Decision Making
• Work Search
Être apte au travail
Choix de carrière
Dév
eloppem
ent
de
com
pét
ence
s
Recherche d’emploi
Maintien en em
ploi
Croi
ssan
ce
prof
essi
onne
lle
Dimensions de l’employabilité
8
Description of LMI Intervention4 weeks of intervention •Using the LMI booklets according to their
identified need in ▫Career Decision Making or▫Work Search
•Method of using LMI booklets:▫Independent▫Assisted▫Information and advice follow-up interviews (2)
week one and and week 3
Career Decision MakingKnow yourselfKnow the labour market
Arrive at a goal and develop an action plan
Work SearchCheck for « fit »
Get ready
Search for work
Secure employment
9
Client Sample• 52 clients between18 and 56 years of age (average = 37 years)• 25 men and 27 women• 83 % had at least a secondary school diploma• The majority worked in industry (eg.: seasonal workers, labourers)• 81 % of clients were unemployed : 19 % were in full or part time
employment
Two methods
Independent Assisted Total
Two kinds of interventions
Career Decision Making
15 17 32
WorkSearch 11 9 20
Total 26 26 52
10
Career Practitioner Sample
• 15 career practitioners• 3 men and12 women•Number of clients per career practitioner• Average = 3,47 (median = 3,87), range: between1 –
13.• Between1 and 3 clients = 12• Between 4 and 6 clients = 0• More than 7 clients = 3
11
Methodology•Post-pre self-assessment ▫At the beginning of an intervention, people do not always
know what they do not know….so there is a tendency to over or under-estimate …..« Knowing what you know now (having completed the intervention), how would you rate your knowledge when you entered the research and how would you rate your knowledge now?»
•Dependent measures▫Knowledge of LMI▫Ability to use LMI▫Personal attributes (confidence and optimism)
12
Principal Findings
1. Clients on average demonstrated significant increases regardless of the method used (independent or assisted)
2. Clients in the assisted group demonstrated greater increases over time than the independent group.
3. Assisted clients in the career decision making group demonstrated greater increases than the other three sub-groups
1. Independent career decision making2. Assisted career decision making3. Independent work search4. Assisted work search
4 sub-groups
13
First Major Finding•Clients demonstrated on average significant
increases regardless of method of intervention (assisted or independent) ▫Between pre and post intervention:
•Overall ability to access and use LMI improved:▫Knowledge re how to use LMI▫Abilities to use LMI and to take action based on
LMI▫Personal attributes: optimism, self-confidence and
by inference, motivation
Sample of Results: Post-pre questionnnaire
0) Ne s’applique pas du tout1) Ne s’applique pas du tout, mais presque2) S’applique, mais tout juste bien 3) La réponse se situe entre tout juste bien et parfaitement bien4) S’applique parfaitement
6/14 itemsNON
Sur ces 6 items (Connaissances = 1, 2, 3, 5; Compétences = 4; Attributs personnels = 6)
Descriptive Results – POST-PREConsidering all 14 items in the questionnaire
Before the intervention After the intervention
• 45 % of responses were not OK [0, 1] • 2 % of responses were not OK [0, 1]
• 55 % of responses were OK[2, 3, 4]
• 98 % of responses were OK[2, 3, 4]
• 8 % of responses were exceptional [4] • 52 % of responses were exceptional [4]
15
Descriptive Results– POST-PRE•All the mean scores on the Before Intervention
responses were in the Not OK range [0,1], with 1 exception (2,23).
•All the mean scores After the Intervention , without exception, were in the very OK or perfectly OK range [score average 3 or more].
•The amount of change was similar across all dimensions of the survey ▫We found the same degree of change for
knowledge, abilities and personal characteristics.
16
17
Second Major Finding
•Clients who were assisted by a career practitioner demonstrated greater change with time
18
•Significant improvement in overall ability to use LMI between Before and After▫These results were consistent in all sub scales
(knowledge, abilities, personal attributes)
Results Overall Score
For the group as a whole:
14 items
Post-Pre
19
Second Major Finding
•Assisted clients showed increased levels of change over time
▫We can consider that having had two information and advice follow-up interviews, although brief, allowed assisted clients to state after the intervention, greater gains in knowledge and skill related to LMI as well as gains in confidence and optimism with respect to their career futures.
20
Information and Advice Interviews• Two interviews of 20-30 minutes each• Three key steps in Information and Advice interviews:▫Re-establish the helping alliance and review the objective and
action plan agreed to▫Give advice and information related to client use of LMI as
appropriate for client need▫Come to agreement on next steps to move client action plan
forward• All Career practitioners reported following these 3 steps ▫ Importance was given to the helping alliance and to agreeing on
the next steps to be followed• Career practitioners used the second step to specifically
respond to the information and action needs of the clients
21
Third Major Finding
• Assisted clients in Career Decision Making reported the highest gains among the four groups
1. Career Decision Making (independent)
2. Career Decision Making (assisted)
3. Work Search (independent)
4. Work Search (assisted)
Results showing both the two types of intervention and the two methods
Final Question-
naire Before
Final Question-
naire After
Two Types of intervention
Two methods n
M(ET)
M(ET)
Time(p)
Method(p)
T x M(p)
Career Decision Making
Independent 15 42,00(17,78)
70,00(13,79)
278,22(< 0,01)
3,58(0,07)
19,69(< 0,01)
Assisted 17 24,65(7,09)
72,94(9,79)
Total 32 32,78(15,69)
71,56(11,73)
Work Search Independent 11 52,82(24,31)
90,27(12,25)
66,35(< 0,01)
0,03(0,86)
0,26(0,62)
Assisted 9 49,00(25,64)
91,44(13,16)
Total 20 51,10(24,32)
90,80(12,34)
22
F
23
Career Decision Making (assisted) (Suggested Interpretation)
• The unique position of clients who are trying to clarify a career direction may promote the asking of clarification questions which could increase the impact of an LMI intervention (Imel, Kerka et Wonacott, 2001)
• The advice given and the individual assistance may have resulted in an increase in the effectiveness of LMI for the individual (Brown et Ryan Krane, 2000)
• « The impact of the help of a career practitioner working with LMI and working directly with a client trying to make career planning decisions appears to be an important finding» (Savard, Michaud, Bilodeau et Arseneau, 2007, p. 163).
Other Results
•No significant differences for: ▫Gender
Men and women responded similarly to both methods of working with LMI
▫Age▫Employment history▫Unemployment history
24
Attribution of change• To what extent would you say that the changes you
reported before and after the intervention were the result of your participation in the research?
Methods
Mainly due to other factors
Partly due to other factors
Don’t know
Partly due to this project
Mainly due to this project
Independent (n=26)
0 0 1 16 9
Assisted (n=26 0 0 0 9 17
Total 0 0 1 25 26
25
26
Attribution of change• 50 % of clients attributed their change to participating
in the LMI research and not to other factors; 48 % attributed their change partially to their participation
•Overall there appear to be no significant differences in the results of the anglophone and francophone samples with the exception that assisted francophone clients reported increased gains over the anglophone clients although both groups reported more positive change in the assisted condition.
Client Engagement (client tracking sheets)
•Clients were very engaged in working with the LMI packages ▫Clients accessed between 4 and 20 distinct LMI resources
and made use of these between 7 and 148 times. ▫On average clients accessed between 4 to 7 LMI resources;
career decision making clients accessed on average 19 resources and work search clients accessed on average 18 resources.
•Most frequently used resources
27
Work SearchLMI Booklets (90 %)Career Cruising (65 %)
Career Decision Making Career Cruising (95 %)LMI Booklets (87 %)Travailler au Canada (87 %)
General Comments from clients
• Increase in confidence▫ I doubted that I would benefit from the study based on past
experience but I became more certain and confident in my own ability to find what I needed. (214)
• Better self awareness and knowledge of possible careers ▫ It was very useful to understand myself better and use this
as a basis for searching for possibilities and making choices. (241)
• Overall a positive experience ▫ The study helped with knowing what direction to go in and I
am sure that I have benefited a lot. I am more confident and pleased to have participated. (231)
▫ This allowed me to get more clear. I think I can continue to use this information as I face future changes. (217)
28
29
Client Criticisms •The program needed to be longer given the
number of resources available. (222)
•All the information I needed was available but I could not take it with me. It would have been better to have it all instead of having to go the the Resource Centre. […]. Working full time made it very hard […]. With a little more help it would have been much easier (242)
Client Criticisms •There was too much information and that made
making a career decision more difficult […] very hard to do this alone (209)
•The information in the booklets was too broad for my region […]. This could discourage certain people (252)
• I feel I am at the same point at the end of the study (237)
30
31
Follow up interviews (1 week and 4 months after intervention)•Week 1: 39 clients (75% of original sample)•4 months: 27 clients (52% of original sample)
▫LMI overall relevant▫Clients overall optimistic and confident▫Clients continued to use LMI but less often
32
Follow up Interviews•Most useful aspects of LMI:▫Self-knowledge▫Job search techniques
•Overall weakness in service delivery:▫Action Planning
33
Follow-up interviews•Most common concerns:▫« too much » information▫Lack of computer skills▫Lack of practitioner assistance especially:
Connecting self knowledge to world of work Addressing employment barriers (confusion,
disappointment)
34
For more information
Réginald Savard, professeur et [email protected], professionnel de [email protected] Bezanson, CCDF, [email protected] du GRDC <http://www.crwg-gdrc.ca