How do we measure the impact of arts and culture on the quality of life of all citizens?
What are the most important criteria in evaluating a successful program?
MODERATOR
Geoffrey Crossick — London, UK
Distinguished Professor of the Humanities
Director, AHRC Cultural Value Project
School of Advanced Study
University of London
PRESENTERS
Mark Stern — Philadelphia, Pennsylvania, USA
Kenneth L.M. Pray Professor of Social Welfare and History
Co-Director, Urban Studies Program
University of Pennsylvania
Christophe Girard — Paris, France
Mayor of the 4th Arrondissement of Paris
SESSION IIASSESSING THE IMPACT
26
SESSION IIASSESSING THE IMPACT
MEASURING THE IMPACT OF ARTS AND CULTURE
Geoffrey Crossick — London, UK
There is a danger in the discussion on how do we evidence or how do we measure the
impact of arts and culture on the quality of life of all citizens. It is important to begin with
some preliminary reflection on the questions that have been presented to us for this session,
in particular to ask why we want to know the answers to them. Why do we want to measure
the impact of arts and culture on the quality of life?
The three reasons, in my view, are:
1) To understand the difference that art and culture makes and why it makes that difference;
2) To seek a more formative way to evaluate—to help people who are practicing in arts and
cultural organizations to do what they are doing better, by understanding the outcomes
of what they do; and
3) To persuade funders that their funding is justified and should be maintained or even
increased.
These three reasons are, of course, connected to each other but they are not the same, and far
too often we fail to make those connections. Instead, we leap to the third of these reasons,
and end up trying to persuade governments, at the local or national level, of the reason why
investing in art and culture matters.
27
The goal becomes advocacy and little more. We end up telling governments what we think
they want to know, and we assume that to mean that arts and culture have a major
economic impact. So, we construct the story that it has major economic impact, and
methodologies are devised to demonstrate that. In the UK, at least, neither the arts
organizations that produce the reports nor the Treasury (i.e. the Finance Ministry) that
receive them, have confidence in the methodologies for measuring economic impact or in
the findings that they yield.
In reality, I don’t believe that governments fund arts and culture because they believe
they’re good for the economy, though they may use the evidence that it benefits the
economy to justify what they’re actually doing for other reasons. And the cultural sector,
mistakenly in my view, goes along with all of that. Surely we can do better?
Changing Our Understanding of the Impact of Arts and Culture
If the advocacy imperative drives our understanding in certain
directions, what happens if we start looking elsewhere?
We could start by asking what arts and culture distinctively
does in the city. So far in our discussions we have focused
primarily on art practice, but for the majority of people in
cities it is above all about arts as an experience, and we must
think about that dimension.
That means highlighting the fundamental ways in which art and culture make a difference
to the good functioning of great cities, because it is here that we find one part of the really
distinctive role that they play.
The important aspect that I wish to highlight might be called reflectiveness, empathy and
difference, and its significance in diverse global cities driven by the dynamics of migration,
diasporas, and diversity.
“...for the majority of
people in cities it is
above all about arts as
an experience.”
28
Engagement with the arts helps you see and understand others differently, something that
is so important in global cities where differences abound and which can as a consequence be
tense, challenging spaces in which to live. Global cities are places characterized by
difference, including differences of language, ethnicity, and religion.
The arts play a fundamental role in helping people to understand their own place in society
and the city, to understand difference, and to appreciate it. It doesn’t remove the tensions
but it can much of the time turn them into something else, something more engaged and
productive. I remember Neil McGregor, Director of the British Museum, recently describing
global cities as Towers of Babel, and that it was cultural institutions and cultural
engagement that were crucial to shaping a world in which the Tower of Babel would not
collapse.
Having considered reflectiveness, empathy, and difference, we might then move on to many
other dimensions where arts and cultural engagement make important contributions to
global cities: dimensions such as mental health, energetic not passive civil societies,
subjective well-being, dynamic innovative environments, and much more.
Those are the kinds of things that we need to be evidencing if we want to move on from the
simplistic story about economic impact. And what makes these aspects and benefits more
exciting and more satisfying to talk about is that, in contrast to economic impact, they relate
to the fundamental experiences that people have when they engage with the arts.
All sorts of activities produce economic impact, but it is much harder to say what else
produces reflectiveness, empathy, and an engaged civil society, as well as the other
benefits to which I’ve just referred.
29
How do we understand and evidence the difference that arts and culture make?
How then are we evidencing that contribution across the various dimensions of impact that
I’ve mentioned? When we broaden our focus, we are freed to look at arts and culture in a
much more expansive way. That means that we should be talking not just about the specific
part of arts and culture that is publicly-funded, but also about the totality of commercial,
subsidized, participatory, and amateur arts.
There is a related danger, which is that our interest in the character of great cities and the
role of culture within them becomes rapidly translated into technical, methodological,
seemingly neutral questions about evaluation, metrics, and methods. These are important
but they are not the whole question. If we take for granted what it is that we are trying to
evaluate, without trying properly to understand the phenomena, we run the danger of
seeking ever more refined ways of measuring something too limited and too simple.
The final point I would like to make is that if we look at the question that we have been
asked, I wonder whether it is right for us to be trying to measure the impact of arts and
culture as opposed to seeking to evaluate it. Nor should we be limiting ourselves to the
evaluation of programs, as if that were the only or even primary form of people’s
engagement.
We need to embrace the much broader ecology of culture in global cities. Measurement can
be a part of that process of evaluation but should not be the whole of it. It takes us back to
the three reasons for evaluating that I highlighted before.
By calling it evaluation rather than measurement, we are signaling that we’re rejecting the
simple methodological hierarchy, one that sees quantification (and often randomized
control trials for specific interventions) as the best way of demonstrating the difference that
is made and all other methods of evaluation as necessarily inferior.
30
There are, for sure, areas where quantification is essential to demonstrate the differences
we’re seeking to evidence. But there are many other areas where it is serious qualitative
research that is what is needed, and where only qualitative evidence can give us the
answers. It should not be regarded as weaker than quantitative evidence, but the passion for
measurement means that it all too often is.
The plural of anecdote is not data.
If our qualitative evidence is to be convincing then it must also be rigorous. People in arts
organizations love telling good stories about the transforming power of what they do, and
these stories are indeed important. But, as someone once observed, ‘the plural of anecdote is
not data.’ If we believe in the power of qualitative (as well as quantitative) evidence, then we
need to move beyond the anecdote to qualitative research that is itself rigorous and
conclusive. In that way we shall avoid having to seek numbers all the time as if they
represented the only form of rigorous evidence.
If we try to understand the difference that arts and culture make better, and not get our
evaluation entirely focused on advocacy, then we might actually find that our advocacy
improves at the same time. I agree that it’s a challenge but I do think we should not simply
get into a position where we seek to do in terms of evaluation and evidence what
governments and funders expect us to do.
We need to develop rigorous methods to evidence those many dimensions of the difference
that arts and culture make that we regard as important, to do so self-critically, and to make
the case for seeing qualitative and quantitative approaches as equally relevant. If we don’t
try to do that, we shall end up diminishing our understanding of what the arts and culture
achieve and the difference they make, and as a consequence, diminishing the case that we
can make to others.
31
NEIGHBORHOOD CULTURAL ECOSYSTEM
Mark Stern — Philadelphia, USA
The Importance of the Neighborhood as a Unit of Analysis
Whenever the topic of impact comes up, it almost invariably leads to the discussion of
individual impacts. How does viewing a painting or listening to a piece of music produce
changes in the individual listener or viewer? The assumption often is that social impacts are
simply the sum of these individual impacts.
This model misses the fact that the production and consumption of culture are essentially
collective, socially constructed processes. In recent years, we’ve become much more aware
of the role of social networks in cultural engagement, but this awareness has had only
limited influence on how we think about impacts.
Neighborhoods are a particular kind of social network, one that exerts a powerful influence
on cultural engagement. We’ve used the idea of a neighborhood cultural ecosystem to
underline the point that the variety of cultural assets present in a neighborhood and their
interaction with one another is the context within which the arts have a social impact.
Breaking Out of the Economic Development Framework
Social Impact of the Arts—University of Pennsylvania (SIAP) was founded with the
intention of developing methods to study the non-economic benefits of the arts, culture, and
humanities. Through much of our existence, we have worked a bit on an ad hoc basis. If we
found some data on public health or racial harassment, we’d examine its relationship to the
arts.
32
Recently, we’ve adopted a more integrated framework for thinking about social wellbeing,
the capabilities approach associated with Amartya Sen and Martha Nussbaum. The
publication of a number of comparative studies of social inclusion, social justice, and social
wellbeing, all of which drew from the capabilities literature demonstrated its empirical
usefulness.
Combining our commitment to the neighborhood as our unit of analysis with capabilities
use of a multi-dimensional view of social wellbeing, we resolved to develop an index of
social well-being at the neighborhood level. Over two years, we worked with a group of
undergraduate and graduate students to pull together data on twelve distinct dimensions of
well-being and study their relationship to the arts and culture. We’ve circulated one of the
two papers we’ve completed on the project.
The scatterplot shows how our measure of morbidity is associated with cultural
participation at the neighborhood level. The work has a long way to go, but we believe that
this framework—one based on a clear conception of social and distributive justice—can
serve as the roadmap for this study. In the next two years, we hope to refine our estimates
for Philadelphia and develop new estimates for several other American cities.
Elitism and Inequality
My third point is more painful. When we began SIAP, we were attracted to the study of
the arts and culture because the map of cultural assets didn’t look like the map of poverty,
crime, HIV/AIDS, or incidents of child abuse. Cultural resources were not so strongly
correlated with other measures of social advantage and we saw that as an opportunity to
leverage cultural assets to improve the lives of socially excluded populations.
Unfortunately, today Philadelphia’s cultural assets are less equally distributed than they
were in the 1990s. Indeed, measures of economic well-being now explain twice as much of
the variance in cultural assets than they did then.
One effect of this increased inequality is the ‘mortality’ rate of cultural organizations.
Many low-income neighborhoods that used to have significant cultural assets—what we call
civic clusters—have lost their resources.
We don’t know exactly what caused this mortality crisis. I suspect some of the factors are
national—like the ‘marketization’ of the nonprofit world—while others may be more local.
We also suspect that global cities, like those that are the subject of this meeting—may be
different from cities like Philadelphia.
We should view the increasing inequality in cultural assets as an existential threat to the
cultural sector. A generation ago, the cultural wars tried to portray the arts as an elitist field
of marginal importance to ordinary Americans.
The reality is that arts and culture is more the province of elites today than they were during
the cultural wars.
34
POLITICAL LEADERSHIP AND CULTURAL INITIATIVES
Christophe Girard — Paris, France
What does culture do to the economy and to the civilization?
In Europe we are facing a very crucial moment and I would like to express some anger and
some fear. Our political leadership, which I am a part of, has a ‘cultural level’ that is
decreasing drastically. Political leaders frequently do not distinguish between culture and
entertainment. They actually believe that it is easier to spend public money on
entertainment rather than helping a small theatre in a district, or a small museum in Paris.
France is considered a country where a high amount of public money is spent on culture
and cultural activities, but it seems now that even France has reached its limit. When the
money for culture was doubled in France, it allowed for new operas to be built, new
libraries to be built, but now, so many years later, we have to spend public money just
preserving these places. Thus, when I was a deputy mayor for eleven years, we carefully
chose places to be shut down, in order for new places to be born. If you just accumulate
places for showcasing of art, what will be the place for the artist to create his art?
When we were elected in 2001 everyone said, of course Paris, the city of lights and lovers,
Notre Dame, the Louvre, there’s not much to do, but I think it is quite opposite. We had to
change that path. So when we look at Paris, for centuries, the culture is along the river, from
east to west.
35
“If you just accumulate places for showcasing of art, what will be the
place for the artist to create his art?”
We have international exchange with cities. In Madagascar, we painted public toilets
because we felt that was the main issue; in Bethlehem, we decided to give benches so
people could sit; in Kabul, we did a public theatre.
I believe that in order to use cultural initiatives for a civil society, there has to be
reorganization. We have to reorganize the mess that permits nothing to bloom, and also,
who wants the world to be wonderless, smooth, and sterilized? Nobody. We need more
artists and not more police.
“We need more artists and not more police.”
KEY ISSUES AND QUESTIONS RAISED IN DISCUSSION
How do we persuade governments that art does matter?
One concern is that all artists and arts organizations want from the government is money.
They want no supervision and want to be left alone. This situation is not that appealing to
governments. It may be the set of conditions under which art thrives, but it’s difficult when
determining what the value proposition is. Other government investments have a
predetermined civic good. So, if money with no deliverables and constraints are the set of
conditions in which the arts community seeks to work, then the case still needs to be made
that these investments do pay off.
Indeterminacy has to be respected. Global cities are magnets of diversity. We have to be
able to say that over time and in the aggregate that these investments do pay off. People
have to be made more comfortable with these concepts of indeterminacy and porosity. It
cannot be seen as individual parts but rather as a genus that does pay off, down the road.
People migrate and create heterogeneous environments. What art does to this value of
diversity is it creates an imaginative environment. This could be a tool for measurement,
assessing the value of art to diversity, and the value of diversity to the creation of art.
Art must be understood as a total ecology.
We may not understand the full roles each part plays, but they are in the system so they are
important. Institutions like museums—rather than recording the attendance of museum
visits, they should look at whether they have reached a cross section of the population of the
city. Success of the institution cannot be determined by figures alone.
37
We need our polar bear.
The environmental movement has managed to get their message across with the image of
the polar bear on a shrinking ice cube. Everyone gets that image. The arts community needs
such a powerful image. We are looking at this the wrong way. The relationship is ordinarily
seen as an authoritative institution versus the receivers—who is the audience, i.e. we see
the audience as the customer. The city has to introduce ‘co-ownership’ of the institution, i.e.
public co-ownership of the institution.
Take the example of the strikes in Paris in 2004 when there were changes in benefits for
artists and technicians. There was almost no festive activity on the street for a few months
and the mayors panicked. They realized that artists bring the humanity and make a city
beautiful. It is important to let people in the process.
The complexity of dichotomies in art practices such as high art and popular
culture need to be considered.
There has been a shift from the conception of graffiti and street art as being purely popular
expression coming from certain neighborhoods and certain communities. At some point it
was assigned value as art, as opposed to public nuisance, because it was commoditized and
displayed in art galleries in a certain way. At some point it became a setting for a
McDonald’s commercial. We must consider the full range of the way that the same
expression can travel between a purely non-market expression component to a high arts
component to an entertainment component—and back and forth.
Citizenship vibrancy is not something that can be quantified. Learning to ‘listen’ to the city
that speaks is also a way of response. A healthy cultural environment is one that is
sustainable or well-balanced, universal, or accessible.
38
In very big and complex global cities there are three main challenges faced by a
policy maker:
1. How do we cover the whole city? With all the programs, tools, funding, and systems?
How do we cover the full diversity? How do we make what we do accessible for all?
This is not only access to consumption but also to ways and means of production.
2. How do we balance supply and demand? All cultural policies focus on arts communities.
There should be a balance between the interests of the cultural organizations and the
community. This means there has to be a change in the way we do things.
3. How do we make culture a priority for a society’s agenda?
Entertainment cannot be defined out.
In many ways expressive impact, consumption versus participation in the entertainment
context, cannot be ruled out as part of the continuum, both from the economic impact and
from the expressive impact as well as from the communal self-definition component. This
should be inserted into the conversation. When we define separately, consumption versus
participation, we describe a lot of art that happens in the entertainment context.
For example, a garage-band playing rock songs—are we considering that arts practice or
not? We need to reign in that complexity.
Having a physical space for the Artist matters as much as a metaphorical artistic
space.
Neighborhoods are critical spaces where such ‘spaces’ become available. Something of the
sort that makes Berlin such an extraordinary space. Berlin is admired for being such a center
for art, but it is forgotten that it started out being a city with a lot of empty and unoccupied
space.
39
What is shrinking in Manhattan, Paris, and other global cities are places for production of art.
One example is the creation of a participant budget in Brazil, in which the local community
decides what city hall should do with the money. Surprisingly, the budgets in Brazil were cut
down considerably. Elected politicians must accept that they are participating in a long-term
process.
It is the nature of global cities that they are very diverse.
We might celebrate the diversity of global cities, but they are also places of conflict. What
arts and culture can do, is not create a ‘harmonious whole’ but create an appreciation across
cultures, which can help that dynamic environment to become more stable.
The digital space must also be considered where coproduction is much more significant than
physical space.
“What is shrinking in Manhattan, Paris, and other global cities
are places for production of art.”
40