Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model

Post on 06-Jan-2016

21 views 0 download

Tags:

description

Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model. www2.uiah.fi/projects/metodi/15b.htm. Kim Dunphy, Program Manager, Cultural Development Network PhD researcher, Deakin University. Overview. The problem: - PowerPoint PPT Presentation

transcript

Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model

Kim Dunphy, Program Manager, Cultural Development NetworkPhD researcher, Deakin University

www2.uiah.fi/projects/metodi/15b.htm

Overview

The problem: •Reframing the intrinsic/instrumental dichotomy•How can qualitative research about outcomes of arts engagement be presented simply?

The solution:•multi-dimensional framework of outcomes•a pictorial model

Case study: youth theatre project, Scared Cool, Ba Futuru, Timor-Leste

The problem

…..the arts sector’s

failure to find a framework that articulates its value in a cohesive and meaningful way, as well as by its neglect of the compelling need to establish a system for collecting evidence around a set of agreed indicators that substantiate value claims (Scott and Soren 2009)

Intrinsic- instrumental value conundrum

A pictorial model of evaluation for arts engagement

Finding a solution

A pictorial model of evaluation for arts engagement

To every public intervention, (at least) four lenses should be applied.

What is impact in terms of

- economic viability

- social inclusion

-environmental sustainability

- cultural vitality

A pictorial model of evaluation for arts engagement

Taking a multi-dimensional approach: Four Pillars of Sustainability (Hawkes 2001)

Image: Marla Guppy

A pictorial model of evaluation for arts engagement

A pictorial model of evaluation for arts engagement

But, ……………..

an emerging theme of enjoyment, pleasure,

where does it fit?

Themes of pleasure, fun, enjoyment

I loved working with Kallista, because she has something new for us (participant, 24 years)

(In attending) I was thinking about myself too. I wanted …. to have a good time- the show was funny

(foreign audience member)

My family loved this drama, they came to watch and they said, its really really good (participant 21 years)

A pictorial model for arts engagement

A pictorial model of evaluation for arts engagement

Role of pleasure in arts engagement

Cook Islanders view the main, and very important purpose of dancing, as creating pleasure or happiness, relating to a world view that the purpose of life is to have fun, anga’anga tamataora, literally ‘to work pleasure’ (anthropologist Alexeyeff, 2009)

‘Fun’ is a major motivator and outcome of participation in dance programs at school and in community settings for young American and Australian children (educational researchers Bond and Stinson, 2001, 2007)

Maori migrants In Melbourne indicated that ‘fun’ was a major motivator for participation in cultural performing groups

(community cultural development researcher Dunphy, 1993) Young people in northern Ireland were motivated by enjoyment of community music making, to engage in programs about peace-building, through which peace-building objectives of host organisation were achieved

(peace-building researcher Pruitt, 2011)

Most commonly mentioned (36%) personal benefits of attending or participating in arts activities include ‘entertainment or fun’ (Ontario Arts Council, 2010)

Taking a multi-dimensional approach: CIV’s five domains

• Healthy Safe and Inclusive Communities• Dynamic Resilient Local Economies• Sustainable Built and Natural Environments• Culturally Rich and Vibrant Communities• Democratic and Engaged Communities

A pictorial model of evaluation for arts engagement

Economic dimension

Civic engagementdimension

Cultural dimensionSocial

dimension

Environmental dimension

PersonalWellbeing

Personal/spiritual

wellbeingdimension

• spiritual • Historic (Throsby 2001, Holden 2006)• expression of communal meanings.

Reclassifying all outcomes of arts engagement into these dimensions

A pictorial model of evaluation for arts engagement

Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002)

• capacity for empathy (McCarthy 2004)• friendship• belonging/ relatedness/ sense of community• safety• social capital- bonding and bridging

Social

dimension

Economic dimension

•skill development •employment•business development•increased wealth

A pictorial model of evaluation for arts engagement

•sense of place•neighbourhood character•connection to the natural world• air quality•water quality•bio-diversity social

•capacity for empathy (McCarthy 2004)•friendship•belonging/ relatedness/ sense of community•safety•social capital- bonding and bridging

environmental

A pictorial model of evaluation for arts engagement

Reframing much of the ‘intrinsic’•aesthetic•symbolic•authenticity •cognitive growth•historic (Throsby 2001, Holden 2006)•expression of communal meanings.•cultural capital- education (Bourdieu) (McCarthy et al 2004)•identity affirmation•respect for diversity

•sense of place•neighbourhood character•connection to the natural world• air quality•water quality•bio-diversity

environmental

cultural

A pictorial model of evaluation for arts engagement

Reframing much of the ‘intrinsic’•cognitive growth•Aesthetic•symbolic•authenticity•spiritual •Historic (Throsby 2001, Holden 2006)•expression of communal meanings.•cultural capital- education (Bourdieu) (McCarthy et al 2004)•identity affirmation•respect for diversity

•active citizenship•membership of local organisations and decision-making bodies•having a say on important issues•engagement in political process

cultural

civic engagement

A pictorial model of evaluation for arts engagement

Personal/spiritualWellbeing

• pleasure, fun• subjective well-being• self-reported health• psychological distress

Civic engagement • pleasure, fun

• subjective well-being• self-reported health• psychological distress

A pictorial model of evaluation for arts engagement

EconomicViability

CivicEngagement

CulturalVitality

SocialInclusion

EnvironmentalResponsibility

PersonalWellbeing

autonomy

autonomyRelatedness

Relatedness

autonomy

autonomyautonom

yautonom

y

Pleasure

autonomyautonomy

auto

nom

yau

tono

my

Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002)

A pictorial model of evaluation for arts engagement

Increased access to economic resources•skill development •employment•business development•increased wealth

EconomicViability

CivicEngagement

CulturalVitality

SocialInclusion

EnvironmentalResponsibility

PersonalWellbeing

autonomy

autonomyRelatedness

Relatedness

autonomy

autonomyautonom

yautonom

y

Pleasure

autonomyautonomy

auto

nom

yau

tono

my

A pictorial model of evaluation for arts engagement

Considering different perspectives

Different stakeholders

•Beneficiaries•Fieldworkers•Project co-ordinators•Manager•Host organisations•Funders•Government

Most Significant Change(Davies and Dart 2005)

Olaf Breuning (2009), Different Perspectives

Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005)

ParticipantsParticipants

Audience MembersAudience Members

Artistic Leader/sArtistic Leader/s

Host OrganisationHost Organisation

FundersFunders

Wider CommunityWider Community

A pictorial model of evaluation for arts engagement

ParticipantsParticipants

Audience MembersAudience Members

Artistic Leader/sArtistic Leader/s

Host OrganisationHost Organisation

Funders

Wider CommunityWider CommunityFamilies and friends not attending

Unfunded activity

Peace-building organisation Ba Futuru

Visiting theatre director, youth co-ordinator, young menteesAudiences of local people; audiences of foreigners

Young performers …

Pride in young person’s achievements

the event was ‘a brilliant PR tool’

that night was a highlight of my life- to see the audience’s response.

I think this show was very important, [....] for human beings to express their feelings

We can [....] change our bad thinking [....] and experience from the past’.

Considering outcomes for all stakeholders in Scared Cool youth theatre project

A pictorial model of evaluation for arts engagement

A pictorial model of evaluation for arts engagement

Intended and unintended outcomes

A pictorial model of evaluation for arts engagement

Intended outcomesBa Futuru’s peace-building mission:

- recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence.

- arts included because of the connection between participatory arts experiences and positive development.

A pictorial model of evaluation for arts engagement

Intended outcomesBa Futuru’s peace-building mission:

- recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence.

- arts included because of the connection between participatory arts experiences and positive development.

A pictorial model of evaluation for arts engagement

Unintended outcomes

Participant Felis:Learning English is important for my future

A pictorial model of evaluation for arts engagement

Positive/ negative outcomes

A pictorial model of evaluation for arts engagement

‘Because the foreigners teach us, that's why I wanted to attend. If the teacher was Timorese, I would not have done this program’. (Participant, 18 years)

‘I am very comfortable when I am with malae (foreigners), because they can understand me. When I ask something, they always answer nicely. When I ask for some help, they will always help me. But East Timor people [….] they never try to analyse what is going on, they just stop in the short term’. (Participant, 19 years)

Positive/ negative outcomes

A pictorial model of evaluation for arts engagement

Positive/ negative outcomes

A pictorial model of evaluation for arts engagement

Unintended

Intended

A pictorial model of evaluation for arts engagement

Depicting intended/unintended; positive/negative outcomes

Expected and unexpected outcomes:

• Comparing between what org expected would happen and what actually occurred

A pictorial model of evaluation for arts engagement

Length of time of change

• Considering short, medium and long term change

A pictorial model of evaluation for arts engagement

Return on Investment

• Comparing what was invested- ‘cost’:• Culturally, economically, socially, personal

wellbeing, civically, environmental

• With ‘benefit’- outcomes in all dimensions

A pictorial model of evaluation for arts engagement

Unintended

Intended

A pictorial model of evaluation for arts engagement

Scared Cool theatre project: outcomes for participants

EconomicViability

CivicEngagement

CulturalVitality

SocialInclusion

EnvironmentalResponsibility

PersonalWellbeing

A pictorial model of evaluation for arts engagement

Unintended

Intended

A pictorial model of evaluation for arts engagement

Depicting intended/unintended; positive/negative outcomes

Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005)

ParticipantsParticipants

Audience MembersAudience Members

Artistic Leader/sArtistic Leader/s

Host OrganisationHost Organisation

FundersFunders

Wider CommunityWider Community

A pictorial model of evaluation for arts engagement

Challenges

• Timeframe for evaluation- does the engagement have an impact (short-term) or outcome (longer term)?

• Does reported change lead to actual change?

• Attribution of outcomes, given lack of ‘control’?

A pictorial model of evaluation for arts engagement

In concluding

Thinking beyond intrinsic-instrumentalto

a multi-dimensional, multi-perspective framework

for understanding outcomes of arts engagement

A pictorial model of evaluation for arts engagement