PowerPoint Presentation

Standing on the shoulders of giantsTristram Hooley (Professor of Career Education)How the evidence base in career development can inform your practicewww.derby.ac.ukwww.derby.ac.uk/icegs www.derby.ac.uk/icegs

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Evidence based practice and policywww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Take a long hard look in the mirrorWhat in your practice is evidence based?What in your practice is not evidence based?What evidence would you like?What do you think works and why?Whose job is to get you the evidence you need?

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs By the end of this session we will haveReviewed the existing literature on the evidence base on career development. Considered the implications of existing research for your practice. Mapped gaps in the literature and identified how filling some of those gaps could enhance your practice.Discussed strategies to gather evidence for your practice, in particular by thinking about how to build alliances with academics and established researchers.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Overviewwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Overviewwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs WellHow do you know?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs KnowledgeKnowledgeEvidencewww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Does it work?Research is specific not general. However there is clear evidence that careers work has a range of impacts.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Key papersWhiston et al.s (1998) meta-analysis of 47 studies identified impacts across all types of career interventions.Bimrose et als (2008) five-year longitudinal tracking study of 50 career guidance clients found that one-to-one guidance interventions were regarded as useful by clients, and that guidance services can support adults to make successful transitions in a turbulent labour market.Vuori et al.s (2012) paper used a randomised control trial to demonstrate the impact of a group intervention on career management skills. Carey & Dimmitt (2012) found that there was consistent evidence of a positive relationship between well-organised school counselling programmes and the educational outcomes of students.And lots more see the reference list at the end.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Overviewwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs What do you do that you think works really well?

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs FeedbackBest technique/approach from each group.Did anything divide you?www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Summarising the evidence base: focus on the individualwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs How are you focusing on the individual?What could you be doing new/additionally/differently?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Summarising the evidence base: support learning and progressionwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs How are you supporting learning and progression?What could you be doing new/additionally/differently?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Summarising the evidence base: ensuring qualitywww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs How are you supporting learning and progression?What could you be doing new/additionally/differently?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs 10 evidence-based principles for the design of lifelong guidance services www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Popular careers ideas with ???? around the evidenceThe idea that the labour market and organisations were once stable and are now boundarylessLearning stylesMBTINLPThe importance of matching personality type to job rolesCareer plansLinkedIn profilesGenerational differencesEtc, etc., etc.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs 23Overviewwww.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs What is evidence?Information that informs practice. This might include LMI, research about career decision making and employer attitudes etc. The available and publicly accessible body of facts that can support propositions about careers work. www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Example 1: Australian LMPAustralian Labour Market Portal http://lmip.gov.au/default.aspx?LMIP

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Example 2: Hyperbolic discounting Given two similar rewards, humans show a preference for one that arrives sooner rather than later (present biased).Would you prefer a dollar today or three dollars tomorrow? Would you prefer a dollar in one year or three dollars in one year and one day?This seems to hold true for career decision making.

See Schoenfelder, & Hantula (2003); and Saunders & Fogarty (2001). www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Example 3: Employers attitudes to disabilityAustralian employers hold relatively positive attitudes regarding individuals with disabilities. However, employers feel less positive about disabled people in the workplace.This impacts negatively on hiring decisions, provision of accommodations and work performance appraisals.

See Burke et al. (2013).

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Implications of these examplesNone of these examples say anything about career development practice (whether it works or what works)But all are suggestive of implications for careers work.How can we engage with such a broad range of literature?

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Strategies for engagementSeek out syntheses and knowledge translation e.g. Running in a Forest, Jockey and the Architect.Commit to learning one thing a week e.g. watch a Ted Talk. Go to more conferencesGet on Twitter and follow some interesting people. Write a blogwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Levels of impact (See Kirkpatrick)www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs ReactionHow do users of services react to them?Use of feedback forms and happy sheets.

Voices of Users (Vilhjlmsdttir et al., 2011) is a summary of the experience of career guidance clients in Nordic countries. It found that the majority of participants were satisfied with the service that they had received and felt that the counsellor with whom they had worked had been supportive and understanding.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs LearningDid the individual learn something as a result of the intervention. If you tested them before and after would something have changed?

Learning for now or later (Kuijpers & Meijers, 2012) found that students career competencies (reflection, exploration, proactivity and networking) were correlated with the presence of a practice based and inquiry based curriculum which allowed them to engage in career conversations. www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs BehaviourHow do individuals behaviour change following an intervention.Can you measure what they do differently?

A career workshop was developed in Switzerland to promote the career choice readiness of young adolescents. In an evaluation of the workshop with 334 Swiss students in the 7th grade, Hirschi & Lge found that three months after the workshop, participants had significantly increased their career decidedness, career planning and career exploration.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs ResultsWhat actually happens as a result of your intervention?Do people get jobs, better qualifications etc.

A report in Northern Ireland (Regional Forecasts, 2008) examined the impact of the Educational Guidance Service for Adults on the Northern Ireland economy. The study used a mix of quantitative and qualitative methods, including detailed analysis of the services client data, to estimate the economic value of the service. This was estimated to be 9.02 net additional tax revenue for every 1 of public money invested. www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Your practiceWhat level of evidence is it based on?Could you evaluate it at the next level up?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Overviewwww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs What evidence would you like to exist?www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Finding evidenceSearch SynthesiseDevelop new research questionsConduct research or commission studiesPublish new evidencewww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Why evaluate?Evaluation enables us to: examine what we dothink about how we can improve it decide on whether it was worth doingprovide others with a summary to help them to understand what was done.

www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs What is evaluation?When people seek to evaluate what they are doing as part of an attempt to learn and improve, they are usually undertaking a formative evaluation, so called because it is undertaken to inform what is done while the activity is still in progress. We would like to find out how to do these things betterWhen people evaluate to make a judgement on the value of a particular activity and to draw out what has been learnt, it is usually a summative evaluation; so called because it attempts to create a summary of what has been achieved and what the impacts have been. We would like to find out how well these things work

www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Discussion: Evaluation dataWhat data do you have which could allow you to make a judgement about the impact of your practice?What level of impact would it describe?

What data could you collect which could allow you to make a judgement about the impact of your practice?What level of impact would it describe?www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Publish or perishToo little evaluation and impact work on careers work is published.

Writing up your evaluation for broader circulation is an important way to support the development of the sector. Self publicationJournal publicationPartnership with academicsUsing external consultants

www.derby.ac.uk/icegswww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Write it downMy main reason for writing is simple: I do not know what I think until I have written it. In conversation one can get away with loose, exploratory thinking, but in writing it down one has to weigh up the arguments and the evidence, and decide what it all means and where one stands. It is hard work, but important; and if published, it adds to the body of knowledge on which others can draw. Tony Watts

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs On working with academics and researchersWhat matters to themTimeMoneyAuthorshipImpactWhat you can give themYour timeFundingAuthorship or co-authorshipDataAccess to interventions and research populationsThe change to impact on practicewww.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Final reflection

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs References and resourcesAustralian Labour Market Portal http://lmip.gov.au/default.aspx?LMIP Bimrose, J., Barnes, S-A. & Hughes, D. (2008). Adult Career Progression and Advancement: a Five-Year Study of the Effectiveness of Guidance, Coventry/London: Warwick Institute for Employment Research/Department for Innovation, Universities and Skills.Burke, J., Bezyak, J., Fraser, R. T., Pete, J., Ditchman, N., & Chan, F. (2013). Employers' Attitudes Towards Hiring and Retaining People with Disabilities: A Review of the Literature. The Australian Journal of Rehabilitation Counselling, 19(01), 21-38.Carey, J. & Dimmitt, C. (2012). School counseling and student outcomes: summary of six statewide studies. Professional School Counseling, 16(2): 146-153.Hirschi, A. & Lge, D. (2008). Increasing the career choice readiness of young adolescents: an evaluation study. International Journal for Educational and Vocational Guidance, 8(2): 95-110.Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU.Kirkpatrick, D.L. (1994). Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler.Kuijpers, M., & Meijers, F. (2012). Learning for now or later? Career competencies among students in higher vocational education in the Netherlands. Studies in Higher Education, 37(4), 449-467.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Need to revise this47References and resources IIRegional Forecasts (2008). Examining the Impact and Value of EGSA to the NI Economy. Belfast: Regional Forecasts.Saunders, R., & Fogarty, G. J. (2001). Time discounting in relation to career preferences. Journal of Vocational Behavior, 58(1), 118-126.Schoenfelder, T. E., & Hantula, D. A. (2003). A job with a future? Delay discounting, magnitude effects, and domain independence of utility for career decisions. Journal of Vocational Behavior, 62(1), 43-55. Vilhjlmsdttir, G., Dofradttir, A.G. & Kjartansdttir, G.B. (2011). Voice of Users Promoting Quality of Guidance for Adults in the Nordic Countries. Oriveden Kirjapaino, Finland: Nordic Network of Adult Learning.Vuori, J., Toppinen-Tanner, S. & Mutanen, P. (2012). Effects of resource-building group intervention on career management and mental health in work organizations: randomized controlled field trial. Journal of Applied Psychology. 97(2): 273-286. Whiston, S.C., Sexton, T.L. & Lasoff, D.L. (1998). Career intervention outcome: a replication and extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45(2): 150-165.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Stuff Ive writtenDent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE.Hooley, T. (2014). The Evidence Base on Lifelong Guidance. Jyvskyl, Finland: European Lifelong Guidance Policy NetworkHooley, T., Hutchinson, J. and Neary S. (2014) Evaluating Brightside's Approach to Online Mentoring. Derby: International Centre for Guidance Studies. University of Derby.Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic.Hooley, T., Matheson, J. & Watts, A.G. (2014).Advancing Ambitions: The Role of Career Guidance in Supporting Social Mobility.London: Sutton Trust.Hooley, T., Mellors-Bourne, R.and Sutton, M. (2013). Early Evaluation of Unistats: User Experiences. Bristol: HEFCE.Marriott, J. and Hooley, T. (2014). Evaluating the Legacy Careers Project. Derby: International Centre for Guidance Studies, University of Derby.Taylor, A.R. & Hooley, T. (2014). Evaluating the impact of career management skills module and internship programme within a university business school. British Journal of Guidance & Counselling, 42(5): 487-499.

www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs www.derby.ac.uk/icegsTristram HooleyProfessor of Career EducationInternational Centre for Guidance StudiesUniversity of Derbyhttp://www.derby.ac.uk/icegs t.hooley@derby.ac.uk@pigironjoe

Blog athttp://adventuresincareerdevelopment.wordpress.com www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs Remember, rememberYour practice is only as good as the evidence that underpins it. There is no point in being really good at doing something that has no impact.Your job and your profession are vulnerable where there is no evidence.There arent many professional career education and guidance researchers. Research takes some skill, but it isnt rocket surgery. www.derby.ac.uk/icegs www.derby.ac.ukwww.derby.ac.uk/icegs