Evidence Review: An evaluation of the impact
of adventure learning and leadership
programmes on young people’s self-esteem,
self-confidence and resilience
A Pro Bono Economics Report
Eugenia Caizzi, Matthew Parish, Samuel Mackley,
Nidhi Arun, Dylan West, Opeyemi Otunuga
August 2018
Contents Executive Summary ................................................................................................................................. 2
Key Findings ......................................................................................................................................... 2
1. Introduction ......................................................................................................................................... 3
Methodology ....................................................................................................................................... 3
2. Programme Differences ...................................................................................................................... 4
3. Characteristics of Participants ............................................................................................................. 6
4. Impact on targeted outcomes ............................................................................................................. 9
5. Methodology Assessment: Data Collection, Measurement and Results ........................................... 12
6. Conclusion ......................................................................................................................................... 16
Annex 1: Research log ........................................................................................................................... 17
Bibliography .......................................................................................................................................... 20
Executive Summary The Dulverton Trust funds a number of charities that offer adventure learning and leadership
programmes for young people to improve behavioural and emotional wellbeing. Pro Bono Economics
are partnering with The Dulverton Trust to create a better understanding of the impact and value of
this funding.
This report provides an overview of the existing evidence base relating to whether such adventure
learning and leadership programmes have a positive impact on young people’s self-esteem, self-
confidence and resilience.
Key Findings • There is a range of empirical evidence that attributes improvements in social and emotional
skills to participation in adventure learning programmes. Furthermore, it is widely recognised
that enhancing these skills results in improved development and wellbeing of young people1.
• The term “adventure learning” covers a wide range of programmes that vary substantially in
their structure, length and type of activity. Some common aspects amongst programmes that
are proven to be effective include: having a variety of activities (as opposed to a single one) and
the involvement of participants as part of a wider group.
• Arguments are made in the literature for why outdoor activities are likely to affect participants
differently depending on characteristics such as age, gender and risk factors related to poor life
outcomes. However, although there is evidence that interventions have led to positive results
for a range of demographic groups, there is insufficient robust empirical evidence to
differentiate conclusions between them and drive programme design.
• There is no consensus as to which of the three targeted outcomes (self-esteem, self-confidence
and resilience) are most significantly impacted by adventure learning programmes. However,
there is significant evidence that such programmes provide improvements in young people’s
wellbeing, including self-esteem, self-confidence and resilience2.
• There are methodological limitations and disparities in measurement techniques that are well
recognised in the existing evidence base. These limitations do not allow definitive conclusions
to be made for many of the issues explored in this report as the results between different
studies become difficult to compare.
• Further research would be strengthened by the development of a clearly defined theory of
change for these types of programmes as well as the adoption of consistent, externally
validated measures of the psychological traits of interest and greater consideration of how to
match participants against suitable control groups.
1 OECD (2018) 2 CUREE (2012)
1. Introduction The Dulverton Trust is a grant-making charity that supports UK charities and CIOs (Charitable
Incorporated Organisations) tackling a range of social issues. They understand the value of
strengthening young people’s wellbeing and in their 2016-2017 Annual Report recognised that they
award most grants (38.67%) under the theme of “youth opportunities”, where they detected the
biggest increase in need following cuts in public funding. Through this, they support a wide range of
activities including adventure training and character building, employability skills and opportunities,
youth social action, improving educational attainment, mentoring and coaching and more.
Pro Bono Economics have partnered with The Dulverton Trust to provide a coherent picture of the
effectiveness of adventure learning and leadership programmes for young people. This report reviews
existing literature to assess the impact that such programmes have on young people in relation to three
psychological traits, self-esteem, self-confidence and resilience, focusing on the importance of
developing young people’s wellbeing. We identify and explore emerging themes that appear to affect
the extent of the effectiveness of adventure learning and leadership programmes in the existing
evidence base:
• Section 2 reviews evidence by different types of adventure learning and leadership
programmes.
• Section 3 reviews the evidence by the different characteristics of participants of adventure
learning and leadership programmes.
• Section 4 evaluates how the three psychological traits are affected by adventure learning and
leadership programmes.
• Section 5 discusses the data collection and research methodologies used in the evidence base.
Finally, section 6 of the report provides some concluding comments, including where evidence needs
to be built, to identify general lessons on data collection and attempts to conclude what has been
identified to be most successful and what least, in terms of intervention.
Methodology The review team searched Google, Google Scholar, EBSCO’s Discovery Service and Taylor & Francis
Online databases utilising a combination of the following keywords to identify relevant studies:
adventure learning programmes, outdoor learning, self-esteem, self-confidence, resilience, young
people.
Papers were assessed for relevance using the following factors:
• Post 2015 studies and systematic reviews were prioritised to provide a more recent update to
the comprehensive study completed by Fiennes et al.3
• Those focused on participants younger than 25 were selected to align with the United Nations
definition of youth unemployment.
• UK studies were prioritised, although the scope was broadened to include other countries such
as New Zealand, Australia and USA where UK-based evidence was limited.
3 Fiennes et al. (2015)
2. Programme Differences
There is no universal definition of adventure learning and leadership programmes. This paper defines
them as “activities which offer challenging or new experiences, often due to being unconventional and
predominantly outdoor”.
This section of the paper explains how these programmes can differ from one another and how they
can affect young people’s self-esteem, self-confidence and resilience. When reviewing the literature, it
is clear that adventure learning and leadership programmes differ for two main reasons. Firstly, the
structure, type or nature of activity; and secondly the length of the activity, with length referring to the
duration of the programme.
Our definition therefore captures a wide range of programmes such as: outdoor education and learning
programmes, residential or day trips to outdoor activity centres, field studies and field work, and
community projects. These programmes differ substantially from one another, which ultimately makes
comparisons across the different programmes extremely difficult.
Each of these types of programme can be further divided into multiple activities. For example, Becker4
studies the effects of one type of adventure learning, outdoor education, on students’ learning, social
and health dimensions, and states that “an all-encompassing definition of outdoor education is scarcely
possible due to different meanings, understandings and practices within various research areas,
countries and cultures”.
There is evidence linking different types of adventure learning programmes to positive outcomes for
self-confidence, self-esteem and resilience. For example, two programmes that lead to increased self-
esteem consisted of participants aged 15 to 19 completing sailing training during a ten-day voyage5.
The programme was conducted on sailing vessels and challenged the participants mentally and
physically through challenges such as living in confined spaces, cooperating with strangers, climbing up
tall masts and completing on-board duties regardless of tiredness, sea sickness or adverse weather.
4 Becker (2017) 5 Scarf et al. (2017)
Key findings:
• There is no universal definition of “adventure learning” with programmes differing by type,
structure or nature of activity and length of activity.
• There is some evidence linking adventure learning programmes to positive effects on
young people’s self-confidence, self-esteem and resilience. However, the evidence around
effectiveness of the programmes is thinly spread and varies significantly.
• Some common aspects amongst programmes that are proven to be effective include
having a variety of activities (as opposed to a single one) and the involvement of
participants as part of a wider group.
• Nevertheless, the variation in programme types as well as the thin spread of literature
hinders comparisons between programmes, meaning definitive conclusions cannot be
drawn on what type of programme is most effective.
Another study6 analyses multiple adventure learning programmes and summarises that a seven-day
rock climbing programme ultimately promoted resilient outcomes for the 4-18 year old participants.
However, to draw more specific conclusions from the literature is especially challenging due to a thin
spread of studies, particularly on the effects of adventure learning or leadership programmes on young
people’s self-esteem, self-confidence and resilience. Even where there are studies covering similar
activities there can be differing conclusions. For example, one particular study7 looks into the effects of
a one-year school gardening project leading to increased self-esteem in primary and secondary school
students. The gardening programme took place over the course of one year with an average of four
hours each week. Another intervention8 again looks into the effects of a one-year gardening
programme on primary school students, also with an average of four hours per week. Whilst being
similar in approach, this intervention lead to growth in self-confidence rather than self-esteem.
Due to these limitations, conclusions around which type of programmes are most effective is not
possible with the literature reviewed. Whilst individual studies may conclude that programmes have
beneficial outcomes for young people’s self-esteem, self-confidence and resilience, comparisons
between these programmes are hindered by the inconsistency of programme types and structures,
along with the differences in programme length.
Despite this, there are common aspects between programmes which emerge throughout the literature.
For example, Gonzalez9 states that increased resilience and self-esteem occurs through a wide range
of adventure-based activities. The paper also concludes that having a range of activities, and the
increased length of activities may lead to these programmes being more effective.
Another common aspect contributing to the effectiveness programmes could be the grouping of
participants in a programme. Scarf et al10 states;
“Study 1 revealed that participants who completed the voyage showed an increase
in self-esteem from the first to the last day of the voyage. Partial correlation revealed
that group belonging made a unique contribution to this change. Study 2 replicated
Study 1 and, further, demonstrated that the relationship between group belonging
and self-esteem was not a function of self-efficacy or group esteem.”
The study attributes factors such as group relationships, group norms, group efficacy and sense of
community for the success of these outcomes.
To summarise, whilst not being able to conclude on which adventure learning or leadership
programmes work best for benefiting young people’s self-confidence, self-esteem or resilience, there
may be common aspects of programmes which increase their effectiveness including having a variety
of activities (as opposed to a single one) and/or the involvement of participants as part of a wider group.
6 Gonzales (2014) 7 Bowker & Tearle (2007) cited in Becker 92017) 8 Sharpe (2014) 9 Gonzalez (2014) 10 Scarf et al. (2017)
3. Characteristics of Participants
The underlying motive of adventure learning programmes is to put students in an environment where
they may encounter problems different to their daily lives. Such exercises test their natural disposition
and help develop their character. However, those belonging to disparate groups gain different things
from the same experience11. It is therefore crucial to consider varying outcomes for different groups
while designing outdoor learning programmes. Research in this area is thinly spread and studies are
narrowly focussed. For example, most papers are based on residential programmes for 11-14 year olds.
There is very little insight into effects on youth of other age groups, disabled or post-trauma. Fewer UK
studies were found on the three impact areas this literature review prioritises (self-esteem, self-
confidence and resilience), which allows us to make scant definitive conclusions.
Age
Evidence from a range of studies suggest that adventure programmes affect different age groups
differently, however there is lack of consensus on scale or even direction of these effects12.
Behavioural issues among 2-11 year olds that disrupt learning might be addressed better by matching
learning preferences in an environment that allows more flexible pedagogies13. Additionally, it has been
identified that positive outdoor learning emerged as a consequence of mixed age grouping during early
childhood years14. Despite teacher concerns regarding safety, the mixed age approach fostered
empathy and responsibility in older children as they mentored their juniors, while younger children
engaged in risk-taking and developed resilience. Secondary school students also have the ability to
develop leadership skills in an outdoor environment by being encouraged to make decisions, build
relationships, self-organise and be self-aware15.
A meta-analysis of self-esteem studies, mostly conducted in Western industrialised nations found that
women’s self-esteem is significantly lower than men’s, with a pronounced progressive drop seen
between 12 to 17 years of age16. Diminished confidence and self-worth among women have been linked
11 Overholt & Ewert (2015) 12 Barry et al. (2017) 13 Waite (2010) 14 Rouse (2015) 15 Karagianni and Montgomery (2018) 16 Clay et al. (2005)
Key findings:
• Arguments are made in the literature for why outdoor activities will affect participants
differently depending on characteristics such as age, gender and risk factors related to
poor life outcomes.
• This means that programmes should ideally be designed according to specific target groups
to achieve successful results. For example, different genders may develop resilience via
different types of activity.
• However, although there is evidence that interventions have led to positive results for a
range of demographic groups, there is insufficient robust empirical evidence to drive
programme design.
to body dissatisfaction during adolescence. Several reports have confirmed that adventure experiences
have a positive effect on teenage girls’ perceptions of themselves. A paper that studied the effects a
coeducational course had on undergraduate women between 20-24 years of age found that they begin
to develop appreciation for their bodies and value for its capabilities. Hence, outdoor programmes that
focus on promoting the concept of ‘self’ among adolescents have knock-on effects on the self-esteem
and self-confidence of its participants17.
It is important to flag that the conclusions drawn thus far cannot be generalised and must only be used
as preliminary analysis. There is a need for further research on the themes highlighted and potential
replication of studies with a larger participant pool.
Gender
Evidence shows that participation in adventure learning programmes can increase resilience. However,
it was found that the extent to which programmatic elements enhance resilience differs by gender.
Overholt and Ewert18 use a two-phase approach to evaluate this. Resilience scores for treatment and
control groups were first compared and questions raised were subsequently explored in an interview
process. Both males and females cited stressful events, uncertainty, physical difficulty and peer
relationships as contributors to resilience development and self-confidence. However, girls were
especially concerned with the physical challenges of expedition, whereas boys emphasised relational
difficulties, such as the ongoing need to debrief and discuss interpersonal issues with their group.
People who are gender-typed will suppress behaviours that they perceive as violating gender standards,
thus limiting their behavioural repertoire in given situations. This dominant discourse may be
challenged by opportunities for females to demonstrate physical strength and technical competency
and for males to express emotion and communicate with fellow group members. Participants who were
able to demonstrate non-gender-typed qualities perceived themselves to be more resilient after the
programme. However, because participation in an outdoor adventure course may be considered
outside the norm of typical feminine gender roles, higher resilience among female participants cannot
be directly attributed to the adventure learning programme itself. Additionally, it has been found that
older girls participate in outdoor activities more frequently than their younger counterparts, which
supports the possibility of a biased participant pool19. While the strength of this conclusion is that it is
based on multiple layers of assessment that value qualitative methods, it is also based on a small sample
size with self-reported differences.
Pre-adolescent girls are usually confident and active in outdoor environments however, as they reach
mid-teenage years, their participation often declines. The results of one20 suggest that several
constraints affect this trend especially lack of opportunity, peer relationships, self-concept and access
problems. Evidence accumulated in this study strongly suggests that outdoor programming builds skills
and confidence in girls, thereby making the outdoor environment less intimidating. Adventure learning
programmes should thus provide girls with an opportunity to learn outdoor skills, include unstructured
time to form positive peer relationships, provide female role models and encourage girls to take up
leadership roles among themselves.
There is a diverse range of opinions with regards to whether single-gender or co-educational adventure
activities are more effective in self-esteem and resilience development. Some argue that co-
educational outdoor programmes are better as they encourage resilience through cross-gender
17 Hovey et al. (2016) 18 Overholt & Ewert (2015) 19 Tsikalas et al. (2015) 20 Culp (1998)
learning21. While on the other hand, others find participation in outdoor programming in girl scouts to
be causally linked to self-esteem22. One study concerning adolescent girls has supporting evidence for
both co-educational and single gender programmes23. Co-educational programmes provide the
opportunity for social mingling and athletic competition while single gender programmes give girls a
chance to learn skills that push them outside of their comfort zone. On balance, the paper supports
single gender programmes more strongly.
‘At-Risk’ Youths
A systematic review24 identified 15 studies reporting the effect of physical activity programmes on the
social and emotional well-being of ‘at-risk’ youths. The activities examined were one of three types:
Outdoor adventure, sport and skill-based or physical fitness programmes. Many of the interventions
resulted in significant positive effects displaying the potential to improve social and emotional well-
being in ‘at-risk’ youths. However, the risk of bias was high in all included studies and more rigorous
trials would be needed to evaluate the effectiveness of these programmes. It is also difficult to identify
the individual effects of the three different types of outdoor programmes.
Wilson and Burnor25 studied the association between a compulsory adventure learning programme for
‘at-risk’ youths and their resilience development. The semester-long programme used adventure
activities as the primary education medium and was an urban continuation of school for students who
had previously dropped out, been expelled or were referred for behavioural or truancy issues. The
results suggest that participants perceived changes in their ability to speak out, form connections with
adults at school and form positive relationships with peers. All three of these are related to protective
factors that could help youths be more resilient against the risk factors that had previously led to their
academic failures. The design of this study does not allow for the determination of causal links. The
methods are qualitative and cannot isolate the influence of the challenge programme apprenticeship
versus other aspects of the adventure education semester and other circumstances in the youths’ lives.
There is much scope for future studies to discern specific types of outdoor education activities that
explicitly impact different target groups. Another area of research worth exploring would be the effect
on students from different socio-economic backgrounds and races. Despite being an area of significant
impact, very few UK-based papers were found on these.
21 Overholt and Ewert (2015) 22 Tsikalas et al. (2015) 23 Culp (1998) 24 Lubans et al, (2011) 25 Wilson & Burnor (2011)
4. Impact on targeted outcomes
This section of the report aims to explore the impact of outdoor education programmes on the three
targeted outcomes: self-esteem, self-confidence and resilience. It focuses on the importance of
enhancing youth’s emotional, social and behavioural skills, aiming to evaluate whether the evidence
base supports the research question of this report. Furthermore, it explores the indirect effects
through which this can be achieved, to attempt a final conclusion on whether adventure learning
programmes drive improved self-esteem, self-confidence and resilience.
It is widely recognised in a vast range of existing empirical evidence that enhancing social and emotional
skills, together with cognitive skills, is proven to result in improvements of the development and
wellbeing of young people. Evidence from the Organization for Economic Cooperation and
Development26 shows that it is internationally acknowledged that such skills are important drivers of
young people’s lifetime success, as well as more generally of important life outcomes, such as better
mental health and wellbeing, maximisation of participation in education, training and employment and
reduction in social and behavioural inequities. Deficits in such a critical stage of personal development
may push young people towards a trajectory whereby they are more likely to experience unwanted
social and behavioural problems, as well as detrimental effects on important academic, work and life
outcomes27.
Self-esteem
Self-esteem is defined as positive or negative evaluations of the self28. Scarf et al.29 conclude that,
following a review of a meta-analysis30, comprising 96 studies, self-esteem was the most consistent
benefit of outdoor adventure learning programmes, which they argue to have a greater impact when
compared to classroom-based approaches. In particular, they explore the indirect effects that mediate
the benefits to self-esteem. In their sail-training programme intervention, for instance, participants are
put into groups and they believe that the increased sense of belonging (or group esteem) in turn
26 OECD (2018) 27 See for example: DuBois, Burk-Braxton, & Tevendale (2002), Orth, Robins, Widaman, & Conger (2014) or Trzesniewski et al. (2006) cited in Scarf et al. (2017). 28 Smith & Mackie (2007) 29 Scarf et al. (2017) 30 Hattie et al. (1997)
Key findings:
• There is no consensus as to which of the three targeted outcomes (self-esteem, self-
confidence and resilience) are most significantly impacted by adventure learning
programmes.
• However, there is significant evidence that they provide improvements in young people’s
wellbeing, of which self-esteem, self-confidence and resilience are part of.
• It is also evident that this is achieved through enhancements in other character
development-type outcomes.
• Further research is required specifically targeting such outcomes in order to draw stronger
conclusions.
enhances young participants’ self-esteem. Similarly, Hovey et al.31 explore the relationship between
adventure learning programmes and body image in females as an indirect effect. In fact, they argue
that greater valuation of a young woman’s body and its capabilities, when being challenged physically,
drives higher self-esteem and a sense of empowerment.
Self-confidence
The focus of self-esteem is on own value and worth, as opposed to self-confidence which is more
specifically defined as trust in one’s ability to achieve some goal or activity32. Evidence from Centre for
the Use of Research and Evidence in Education33 identified self-confidence as one of the most
frequently reported student outcomes following their investigated outdoor learning programme. Self-
confidence, however, was associated again with other factors such as overcoming fears and worries
about specific activities, or simply about being apprehensive of going away from home. This study
identifies that offering different levels of participation, for example from surfing to just paddling, fosters
a greater sense of self-confidence in terms of being able to successfully complete an activity.
Furthermore, it also pushes young people to set themselves more difficult goals, and consequently
achieving them enhances their confidence further.
Resilience
Although defined in a number of ways, resilience, is an individual’s ability to successfully cope with
adversity34. It is also widely defined as the quality of being able to return quickly to a previous good
condition after resisting stressful situations. Ewert and Yoshino35 recognise that there are several
studies they reviewed36 that do not recognise any impact on levels of resilience following outdoor
programmes. However, the result from their own study suggest that their examined intervention
(adventure-based expedition) was effective in enhancing levels of resilience. Furthermore, they identify
six specific components associated with the adventure education that facilitate the improvement of
levels of resilience:
• perseverance,
• self-awareness,
• social support,
• confidence,
• responsibility to others,
• achievement
This is somewhat confirmed by CUREE37 as it reports children emphasising on perseverance and “not
giving up”, when asked if they felt more resilient after the adventure programme. Lastly, Richardson’s38
resiliency model posits that individuals begin with a state of biopsychospiritual homeostasis — in other
words, any state where an individual has adapted to his or her situation in life. From here, a disruption
is experienced, which is followed by a phase of reintegration. Most research papers in the area view
self-confidence and self-esteem to be a consequence of resilience developed in an unfamiliar situation.
31 Hovey et al. (2016) 32 Judge et al. (2002) 33 CUREE (2012) 34 Pecillo, Malgorzata (2016) 35 Ewert & Yoshino (2016) 36 For example Skehill (2001) 37 CUREE (2012) 38 Richardson (2002) cited by Overholt & Ewert (2015)
However, there is a major gap in literature documenting the process of reintegration for different
character groups post an adventure learning intervention.
Combining existing literature and the evidence base in this research area, there is no consensus as to
which of the three targeted outcomes (self-esteem, self-confidence and resilience) are most
significantly impacted by such programmes. It also is evident that the choice of outcomes measured is
thinly spread. Fiennes et al.39 outline that the outcomes measured are mainly ‘character development-
type’ (examples include teamwork, communication skills etc.), as opposed to educational ones.
However, there is little consistency in the methods of analysis of such outcomes and in measuring the
extent to which they are affected. As a consequence, this poses substantial limitations for the purpose
of comparison, as well as when attempting to evaluate the impact on targeted and specific
psychological outcomes.
However, a general conclusion can be drawn: there is substantial evidence across a variety of studies
that suggests that adventure learning programmes offer consistent positive effects on young people’s
wellbeing, under which the three traits analysed in this report fall40. It is also clear that all the studies
identify that this is often achieved indirectly – the end-result of increased self-esteem, self-confidence
and resilience is achieved through strengthening of other traits such as group belonging, perseverance
or self-awareness.
Finally, this conclusion supports the scope for a more comprehensive and consistent evidence base and
evaluation of the development of specifically targeted social, psychological and emotional skills for
young people, as a result of an adventure learning programme. More high-quality assessments of the
character-development outcomes, with further exploration of key indirect effects are needed to
attempt to find a definite answer to the research question and to ensure the full potential of the
programmes can be achieved. Perhaps introducing a standardised unit of measurements of such
outcomes could lead to more reliable and valid conclusion on what traits benefit the most.
39 Fiennes et al. (2015) 40 See, for example, CUREE (2012)
5. Methodology Assessment: Data Collection, Measurement and
Results
When reviewing the existing evidence base of adventure learning and leadership programmes on young
people’s self-esteem, self-confidence and resilience, a number of limitations surrounding data
collection and measurement were clear to see. The recurrence of these limitations throughout the
studies and systematic reviews result in a lack of robustness and a largely incomparable evidence base.
Whilst this can be seen in a negative light, it is important to acknowledge these limitations in order to
move forward in improving the evidence base.
Some of these limitations in the evidence base could be made less limiting by improving methodology,
others are inherent in the type of data required to measure the effectiveness of adventure learning and
leadership programmes on young people’s self-confidence, self-esteem and resilience. Nevertheless,
both remain useful to be aware of.
Becker et al.41 highlight a number of methodological limitations of the qualitative studies looked at in
their systematic review. They note a number of reasons for giving the qualitative studies they looked at
low ratings including;
“Insufficient information about the influence of the researcher on the observed or
interviewed participants, and vice-versa; … and an inappropriate connection
between the conclusions and the analyses”
Despite Becker’s review including a wide range of studies in terms of country of origin, type of
programme and outcome measured, the limitations look to be consistent throughout the field. In
addition to the aforementioned limitations, others included selection bias, over-reliance on self-report
measures, small sample sizes, inconsistency in measurement, absence of a Theory of Change and failure
to measure long term impacts.
Selection bias
There was often poor level of similarity between control and treatment groups in studies, which
subjects results to selection bias. This is likely to be characteristic of programmes that require
volunteers to serve as a treatment group, as one could reasonably expect the treatment group to differ
from the control in personality or attitude. Differences in attitude or personality might be captured in
measures of self-esteem, self-confidence and resilience. For this reason, observed positive effects
41 Becker et al. (2017)
Key Findings:
• There is inconsistency in the way data is measured in the existing evidence base.
• This inconsistency results in an inability to compare studies and therefore draw meaningful
conclusions.
• The existing evidence base also suffers from a number of recurring limitations in data
collection, measurement technique and methodology resulting in a lack of robustness.
• Ultimately, the current evidence base is not strong enough to base recommendations for
favourable adventure learning and leadership programmes – further research is required.
might not be as a result of the adventure learning and leadership programme. Instead, it could be
caused by differences between treatment and control groups.
A lot of the studies looked at are aware of this limitation42. Showing an awareness to the problem, but
with no attempt or ability to change the sampling methodology, possibly caused by a lack of resource
or another viable solution. An unequal distribution of gender between control and treatment groups
could also result in selection bias43. With preliminary analysis in that supports the idea that males and
females are affected differently by adventure learning programmes as explored previously in this
review44.
Some studies do address this issue and mention the matching process in their method45, but the
evidence is thinly spread. Studies should seek to match control and treatment groups as similar as
reasonably possible if they hope to report an accurate impact of the programme in question.
Over-reliance on self-report measures
Methods of measuring the effect of adventure learning on self-esteem, self-confidence and resilience
often place an over-reliance on self-report measures, namely a questionnaire given to participants
before and after the programme46. Whilst self-report measures are fundamental in understanding
psychological changes i.e. effects on self-esteem, self-confidence and resilience, there are many
limitations to be aware of – some of which can be made less limiting. However, many studies over-rely
on self-report measures and do not detail if/how they have attempted to make self-report less limiting,
and therefore make it hard to assess robustness.
Most of the limitations centre on participants having to assess themselves, which creates subjectivity.
Barry et al.47 touch on this, stating “[self-report measures] do not rely on standardized outcome
measures and are not subject to external verification”. Similar conclusions are drawn by Overholt and
Ewert’s48 study which states “when using a pre–post survey instrument researchers must take into
account the ways in which participants assess themselves”. Hoskin49 outlines some other limitations of
self-report measures, such as people interpreting questions differently and interpreting the scales on
which they answer differently too. He also mentions the presence of response bias, meaning some
participants give consistently positively or negatively bias responses throughout their questionnaire. As
previously mentioned self-report measures are fundamental in understanding the effect of adventure
learning and leadership programmes on young people’s self-esteem, self-confidence and resilience.
Nonetheless, it is important to consider whether the adoption of standardised, validated measures of
different psychological traits would improve the robustness of the evidence base.
Small sample sizes
A number of the studies reviewed relied on small samples; often fewer than one hundred individuals
are tracked in treatment and control groups collectively. In Hovey’s50 study only 28 students took part
42 See for example Scarf et al. (2017) 43 Scarf et al. (2017) 44 See for example Overholt and Ewert (2014) 45 Ewert and Yoshino (2011) 46 See for example Overholt and Ewert (2014), Scarf et al. (2017) and Furness et al. (2017) 47 Barry et al. (2017) 48 Overholt and Ewert (2014) 49 Hoskin (2012) 50 Hovey (2016)
in the programme. Whether for cost reasons or operational reasons it seems that studies often lack the
resource to be able to collect a large sample.
Small sample sizes can undermine the findings and conclusions from research. As a large limitation it
can decrease the statistical significance of the results and the level of confidence at which you can claim
the effect on the sample is an accurate estimate of what the effect would be on the population. This is
recognised in Overholt and Ewert’s study which demotes itself to “preliminary analysis”51 for this
reason.
The problem of small sample size can also be exacerbated by attrition. Participants have a propensity
to drop out of adventure learning programmes, as those dissatisfied with the experience have no
obligation to partake in programmes for the entire duration. This can not only introduce bias into the
results but would reduce an already meagre sample size. An example of this can be found in Ewert and
Yoshino’s study, from an initial sample of 85, “after screening the data for incomplete questionnaires,
out-of-range scores, or response sets, two unmatched responses were removed, resulting in a usable
sample size of 55’52.
Inconsistency in measurement
Due to the qualitative nature of the research, there are a lot of differences in the way that data is
measured. This inconsistency makes it very difficult to compare studies’ results with one another. Barry
et al53 pick up on this, reporting “due to the heterogeneity of the evaluation studies identified, a
quantitative synthesis of the findings from the intervention studies reviewed was not possible”. This
inconsistency in data measurement means that even in rare situations where studies have had the exact
same focus, the data is incomparable due to difference in measuring technique.
Absence of a Theory of Change
Outlining a theory of change is well accepted best practice in impact evaluation and has been
recognised in the specific context of adventure learning by Fiennes et al.54 It appears from the existing
evidence base for the effects of adventure learning that logic models or theories of change are
unutilized and rarely mentioned explicitly, if at all. It may be possible that a logic model is used but is
simply not mentioned in published reports, however, the lack of such a framework may be a factor
contributing to the weakness of the evidence base and the lack of robustness in the conclusions/results
drawn from studies in this area.
Failure to measure long term impacts
The studies reviewed in this paper often only considered the effects measured from adventure learning
programmes in the short term. Given that impacts from adventure learning may not be sustained in
the long run, a report only tracking short run changes to self-esteem, self-confidence and resilience
would fail to reflect what might be a depreciating effect. This supports the conclusion of Lubans et al.55:
“as none of the studies included long-term follow-ups, it remains untested whether the benefits
associated with participation in physical activity programmes are sustained once youth return to their
daily routines”.
51 Overholt and Ewert (2014) 52 Ewert and Yoshino (2015) 53 Barry et al. (2017) 54 Fiennes et al. (2015) 55 Lubans et al. (2015)
Whilst this is highlighted as a limitation, it is recognised that there are a number of difficulties in
measuring long term impacts, for example in defining a sufficient length of time after the participation
in a given programme which could be used to categorise a “long term” measurement.
Clearly, a significant number of limitations are present throughout the existing evidence base. As
previously mentioned, whilst this may seem negative, acknowledging these limitations will prove useful.
It is clear that improvements need to be made, of which most studies are aware. The main
recommendation in terms of this section is to move towards creating consistency in the way data is
measured to facilitate comparability, of which a Theory of Change may be useful to develop. In addition
to this some general lessons should be taken surrounding selection bias, over-reliance on self-report
measures and the existence of small sample sizes. A lot of our recommendations for future research
align with systematic reviews already in the existing evidence base, which further adds to a call for the
evidence base to be more robust as well as being consistent in measurement. In doing this it will allow
meaningful comparison between studies.
6. Conclusion There is substantial evidence across a variety of different studies that suggests adventure learning
programmes offer positive effects on young people’s wellbeing. However, a significant number of
limitations in methodology are present throughout this evidence base. Consequently, it is not possible
to draw a robust conclusion on what works best in terms of the effectiveness of such programmes as
an intervention on improving the three targeted outcomes: young people’s self-esteem, self-
confidence and resilience.
Inconsistency in the evidence base represents a constraint that also does not allow comparisons to be
made. In fact, different studies explore different types of programmes, of different length and with
different groups of participants. It is evident that such differences have varying impacts on the
effectiveness of the programmes. Therefore, only common characteristics of the programmes can be
identified to be fruitful.
Furthermore, the evidence base does not provide a clear consensus as to which of the three targeted
outcomes (self-esteem, self-confidence and resilience) are most significantly impacted by such
programmes. However, there is significant evidence that adventure learning programmes provide
improvements in young people’s wellbeing, of which self-esteem, self-confidence and resilience are
part of. This is achieved through indirect effects on other character development-type outcomes.
In order to draw clearer conclusions on whether adventure learning and leadership programmes
improve young people’s self-esteem, self-confidence and resilience, further research needs to be
conducted targeting the three outcomes specifically. Consistency in measurement and a method that
allows controlled outcomes to be tested against variables such as participants, type of the programme
and participants would thus improve comparability. This would provide a better understanding of what
exactly, within adventure learning and leadership programmes, works best.
Annex 1: Research log Ref. No
Title of Paper Authors Year of Publication
Relevance
1 The Existing Evidence-Base about the Effectiveness of Outdoor Learning
C. Fiennes, E. Oliver, K. Dickson, D. Escobar, A. Romans, S. Oliver
2015
2 Social Psychological Benefits of a Wilderness Adventure Program
T. Paxton, L. McAvoy
2000
3 The effects of an adventure based program: Mechanisms promoting resilient outcomes among youth in transition from childhood to adulthood
A.Gonzalez 2014
4 Evaluation of Learning Away Hypothesis 9: Resilience, self-Confidence and wellbeing
Centre for the Use of Research and Evidence in Education
2012
5 Review of Research on Outdoor Learning M. Rickinson 2004
6 The Influence of short term adventure-based experiences on levels of resilience
A.Ewert, A.Yoshino
2011
7 Gender Matters: Exploring the process of developing resilience through Outdoor Adventure
J.Overholt, A.Ewert
2015
8 Review: A systematic review of the impact of physical activity programmes on social and emotional well-being in at-risk youth
D. Lubans, R. Plotnikoff, N. Lubans
2011
9 A Review of the Evidence on the Effects of Community-based Programs on Young People’s Social and Emotional Skills Development
M. Barry, A. Clarke, S. Morreale, C. Field
2017
10 Effects of Regular Classes in Outdoor Education Settings: A Systematic Review on Students’ Learning, Social and Health Dimensions
C. Becker, G. lauterbach, S. Spengler, U. Dettweiler, F. Mess
2017
11 Predictors of Change in Body Image in Female Participants of an Outdoor Education Program
K. Hovey, J. Foland, J. Foley, M. Kniffin, J. Bailey
2016
12 Putting research into practice; or putting practice into research in sail training
H. Prince, E.Fletcher
2018
13 Satisfying psychological needs on the high seas: explaining increases self-esteem following an Adventure Education Programme
D. Scarf, S. Kafka, J. Hayhurst, K. Jang, M. Boyes, R. Thomson, J. Hunter
2017
14 Maximising Potential: The Psychological Effects of the Youth Development Programme Project K
K. Furness, M. Williams, J. Veale, D. Gardner
2017
15 The impact of outdoor youth programs on positive adolescent development: Study protocol for a controlled crossover trial
I.Williams, L. Rosec, C. Olssona, G. Patton, N. Allen
2018
16 Promoting Intrapersonal Qualities in Adolescents Evaluation of Rapport's Teen Leadership Breakthrough Program
Y. Hindes, K. Thorne, V. Schwean, A. Mckeough
2008
17 Children and Youth Services Review A. Parkhill, C. Deans, L. Chapin
2017 Moderately
18 Leadership and Adolescent Girls: A Qualitative Study of Leadership Development
M. Hoyt, C. Kennedy
2018 Moderately
19 The Impact of Leadership Programme on Self-Esteem and Self-Efficacy in School: A Randomized Controlled Trial
M. Wong, T. Lau, A. Lee
2012
20 Body image and self-esteem among adolescent girls: Testing the influence of sociocultural factors
D. Clay, L. Vignoles, H. Dittmar
2005 Moderately
21 Losing our way? The downward path for outdoor learning for children aged 2–11 years
S. Waite 2010
22 Adolescent girls and outdoor recreation: A case study examining constraints and effective programming
R. Culp 1998
23 Association between an adventure education apprenticeship and at-risk urban youths’ resilience
J. Wilson, A.Burnor
2011
24 Developing leadership skills among adolescents and young adults: a review of leadership programmes
D. Karagianni, J. Montgomery
2018
25 Mixed-age grouping in early childhood–creating the outdoor learning environment
E. Rouse 2015 Moderately
26 Girls’ Challenge Seeking: How Outdoor Exposure Can Support Girls in Taking Positive Risks
K. Tsikalas, K. Martyn
2015
27 A 10 –day developmental voyage: converging evidence from the studies showing that self-esteem may be elevated and maintained without negative outcomes
S. Kafka, J. Hunter, J. Hayhurst, M. Boyes, R. Thomson, H. Clarke, A. Grocott, M. Stringer, K. O’Brien
2012 ×
28 Bandura goes outdoors: Role of Self-efficacy in the outdoor leadership development process
D. Propst, R.Koesler
1998 ×
29 Diverse aims, challenges and opportunities for assessing outdoor learning: a critical examination of three cases from practice
S. Waite, O. Rutter, A. Fowle, A. Edward-Jones
2017 ×
30 Outdoor adventure education (OAE) in higher education: characteristics of successful university degree programmes
T. Potter, T. Socha, T. O’Connell
2012
31 Outdoor Leadership –The Last Male Domain?
N. Saunders, B.Sharp
2002
32 Independent thinkers and learners: a critical evaluation of the ‘Growing Together Schools Programme
D. Sharpe 2014
33 Gardening as a learning environment: A study of children’s perceptions and understanding of school gardens as a part of an international project
R. Bowker, P. Tearle
2007
34 The dangers of self-report R. Hoskin 2012
35 Social and Emotional Skills – Well-being, connectedness and success.
OECD 2018
36 Evaluation of Learning Away: Hypothesis 9: Resilience, self-confidence and wellbeing.
CUREE (Centre for the Use of Research and Evidence in Education)
2012
37 What is Subjective Well-Being? Understanding and Measuring Subjective Well-Being
Albuquerque, B. 2010
38 "The concept of resilience in OSH management: A review of approaches"
Pecillo, Małgorzata
2016
39 Smith, E. R.; Mackie, D. M. Social Psychology
2007
Bibliography • Albuquerque, B. (2010). What is Subjective Well-Being? Understanding and Measuring
Subjective Well-Being. Available: http://positivepsychology.org.uk/subjective-well-being/. Last
accessed 30th April 2018.
• Barry, M.M., Clarke, A.M., Morreale, S.E. and Field, C.A., 2018. A Review of the Evidence on
the Effects of Community-based Programs on Young People’s Social and Emotional Skills
Development. (Date Accessed: 22nd March 2018). Available from:
https://link.springer.com/article/10.1007/s40894-017-0055-2
• Becker, C. 2017. Effects of Regular Classes in Outdoor Education Settings: A Systematic
Review on Students’ Learning, Social and Health Dimensions. International Journal of
Environmental Research and Public Health. [Online].14(5), p.485. (Date accessed: February
2018). Available from: http://www.mdpi.com/1660-4601/14/5/485/htm
• Bowker, R; Tearle, P. 2007. Gardening as a learning environment: A study of children’s
perceptions and understanding of school gardens as a part of an international project.
Learning Environments Research. [Online]. 10(2), p.83-100. [Date accessed: February 2018].
Available from: https://link.springer.com/article/10.1007%2Fs10984-007-9025-0
• Clay, D., Vignoles, V. L., & Dittmar, H. (2005). Body image and self-esteem among adolescent
girls: Testing the influence of sociocultural factors. Journal of research on adolescence, 15(4),
451-477.
• Culp, R. H. (1998). Adolescent girls and outdoor recreation: A case study examining
constraints and effective programming. Journal of Leisure Research, 30(3), 356-379.
• CUREE (2012) Evaluation of Learning Away: Hypothesis 9: Resilience, self-confidence and
wellbeing. London: Paul Hamlyn Foundation. Available from: http://learningaway.org.uk/wp-
content/uploads/Hypothesis-9-Resilience-self-confidence-and-wellbeing-final2.pdf
• Dulverton Trust (2017): Annual Report and Accounts 2016/17 Available at:
https://www.dulverton.org/wp-content/themes/html5blank-stable/pdf/Annual-Report-2016-
17.pdf
• Ewert, A., Yoshino, A. (2011) The influence of short-term adventure-based experience on
levels of resilience, Journal of Adventure Education and Outdoor Learning, 11:1, pp. 35-50
Available at: http://dx.doi.org/10.1080/14729679.2010.532986
• Fiennes, C., Oliver, E., Dickson, K., Escobar, D., Romans, A. and Oliver, S., 2015. The existing
evidence-base about the effectiveness of outdoor learning. Institute of Outdoor Learning,
Blagrave Trust, UCL & Giving Evidence Report. (Date Accessed 19th March 2018) Available
from: https://www.outdoor-learning.org/Portals/0/IOL%20Documents/Research/outdoor-
learning-giving-evidence-revised-final-report-nov-2015-etc-v21.pdf?ver=2017-03-16-110244-
937
• Gonzalez, A. 2014. The effects of an adventure based program: Mechanisms promoting
resilient outcomes among youth in transition from childhood to adulthood. Undergraduate
Honors Theses. [Online]. 106. Date accessed: February 2018]. Available from:
https://scholar.colorado.edu/honr_theses/106/?utm_source=scholar.colorado.edu%2Fhonr_
theses%2F106&utm_medium=PDF&utm_campaign=PDFCoverPages
• Hovey, K,. Foland, J., Foley, J. T, Kniffin M., Bailey J.: (2016) Predictors of Change in Body
Image in Female Participants of an Outdoor Education Program. Journal of Outdoor
Recreation, Education and Leadership Vol 8 No 2, pp 200-208
• Judge, Timothy A.; Erez, Amir; Bono, Joyce E.; Thoresen, Carl J. (2002-09-01). "Are measures
of self-esteem, neuroticism, locus of control, and generalized self-efficacy indicators of a
common core construct?". Journal of Personality and Social Psychology. 83 (3): 693–710.
doi:10.1037/0022-3514.83.3.693. ISSN 1939-1315.
• Karagianni, D., & Jude Montgomery, A. (2018). Developing leadership skills among
adolescents and young adults: a review of leadership programmes. International Journal of
Adolescence and Youth, 23(1), 86-98.
• Kafka, S., Hunter, J.A., Hayhurst, J., Boyes, M., Thomson, R.L., Clarke, H., Grocott, A.M.,
Stringer, M. and O’Brien, K.S., 2012. A 10-day developmental voyage: Converging evidence
from three studies showing that self-esteem may be elevated and maintained without
negative outcomes. Social Psychology of Education, 15(4), pp.571-601.
• Lubans, D.R., Plotnikoff, R.C. and Lubans, N.J., 2012. A systematic review of the impact of
physical activity programmes on social and emotional well-being in at-risk youth. (Date
Accessed: 22nd March 2018). Available from:
https://onlinelibrary.wiley.com/doi/full/10.1111/j.1475-3588.2011.00623.x
• OECD. (2018). Social and Emotional Skills – Well-being, connectedness and success. Available:
http://www.oecd.org/education/school/UPDATED%20Social%20and%20Emotional%20Skills%
20-%20Well-being,%20connectedness%20and%20success.pdf%20(website).pdf Last
accessed: 30th April 2018
• Overholt, J.R. and Ewert, A., 2015. Gender matters: Exploring the process of developing
resilience through outdoor adventure. Journal of Experiential Education, 38(1), pp.41-55.
• Pecillo, Małgorzata (2016). "The concept of resilience in OSH management: A review of
approaches". International Journal of Occupational Safety and Ergonomics. 22 (2): 291–300.
doi:10.1080/10803548.2015.1126142. PMC 4867880. PMID 26652938.
• Propst, D.B. and Koesler, R.A., 1998. Bandura goes outdoors: Role of self-efficacy in the
outdoor leadership development process. Leisure Sciences, 20(4), pp.319-344.
• Rickinson, Dillon, Teamey, Morris, Young Choi, Sanders, Benefield, 2004: A review of Research
on Outdoor Learning. National Foundation for Educational Research and King’s College
London. Available at: https://www.field-studies-
council.org/media/268859/2004_a_review_of_research_on_outdoor_learning.pdf
• Rouse, E. (2015). Mixed-age grouping in early childhood–creating the outdoor learning
environment. Early Child Development and Care, 185(5), 742-751.
• Smith, E. R.; Mackie, D. M. (2007). Social Psychology (Third ed.). Hove: Psychology Press. ISBN
978-1-84169-408-5.
• Sharpe, D. 2014. Independent thinkers and learners: a critical evaluation of the ‘Growing
Together Schools Programme’. Pastoral Care in Education. [Online]. 32(3), p.197-207. [Date
accessed: February 2018]. Available from:
https://www.tandfonline.com/doi/abs/10.1080/02643944.2014.940551
• Scarf, D; Kafka, S; Hayhurst, J; Jang, K; Boyes, M; Thompson, R; Hunter, J. 2017. Satisfying
psychological needs on the high seas: explaining increases self-esteem following an
Adventure Education Programme. Journal of Adventure Education and Outdoor Learning.
[Online]. 18(2), p.165-175. [Date accessed: February 2018]. Available from:
https://www.tandfonline.com/doi/pdf/10.1080/14729679.2017.1385496?needAccess=true&
• Tsikalas, K. and Martin, K.L., 2015. Girls' Challenge Seeking: How Outdoor Exposure Can
Support Girls in Taking Positive Risks. Afterschool Matters, 21, pp.1-10.
• Waite, S. (2010). Losing our way? The downward path for outdoor learning for children aged
2–11 years. Journal of Adventure Education & Outdoor Learning, 10(2), 111-126.
• Wilson, J. D., & Burnor, A. C. (2011). Association between an adventure education
apprenticeship and at-risk urban youths’ resilience. World leisure journal, 53(4), 255-268.