Evaluation of Tourist Assessment Program in Minnesota

RB & LT

Both Introduce

© 2014 Regents of the University of Minnesota. All rights reserved. 1

RB

Cut if 20 minutes only

(Total 3 minutes) Pair, 1 minute for each person to talk.

Summary statement – understanding the most meaningful impacts of the Tourism Assessment Program was what we were trying to learn as well in conducting this evaluation. To explain what some of these impacts were, we have the following topics for today


RB


RB

Tourism industry and funder focus : on return on investment based outcomes‐ ( a silver bullet solution)

Importance‐Performance Model (IPA) is commonly used in many industries to assess and modify management policies. When used, the model organizes data collected via customer satisfaction surveys into two scored categories: 1) how important a particular service attribute is to the consumer, and then 2) how well the service attribute performed according to the consumers expectation. Martilla and James (1977)

In considering the sustainable development of tourism destinations, Ritchie and Crouch (2003) go onto suggest that a destination’s marketing performance is based on more than two dimensions. “Whereas comparative advantages involve [comprehension of] the resources available to a destination, competitive advantages relate to a destination’s ability to use these resources effectively over the long term.”

Developers and Communities focus :on bridging Different perceptions of what constitutes “community assets” to overcome development barriers

Emery, Fey & Flora (2006) suggest that supporting community development with a lens like the Community Capitals Framework, can help development practitioners not only identify their assets, but also start to map out relationships that exist between each asset category that can then be monitored for change or opportunities for development.

Robinson & Meikle‐Yaw (2007) found that in their developmental work with communities in the Delta region of Mississippi “context and the action do "affect" one another. For example, by purposely mixing African American and White residents in workgroups , they created a discussion environment that was the exact opposite from the existing social and political context which has been competition and conflict between two separate groups.

Chazdon and Paine (2014) also point out that evaluation can be used to convince stakeholders and authorizers what outcomes are truly important (example social capital).


RB

Cousins and Whitmore (1998)

Transformative – wordel

To support political empowerment, emancipation, and social justice. Ultimate decision making and control is by beneficiaries

Practical ‐ puzzle

To support program refinement & problem solving. Decision making is balanced between the researcher and beneficiaries

For this evaluation of Minnesota’s Tourism Assessment Program we used a practical approach to participatory program evaluation. In continuing to refine Extension’s programs to meet the needs of diverse communities however, we acknowledge that opportunities exist to reframe program evaluation in the transformative context as well.


LT

The Tourism Assessment program is a Participatory Action Program

Liz – Pass around Community Tourism Development Manual.

Three‐part data collection in partnership with communities:

*Inventories of community assets that can be mobilized for tourism development

*Industry and academic expert visit assessments of community

*Facilitated community SWOT discussion to identify key developmental opportunities in open community meetings

Final written report that summarizes program findings & recommendations: ‐‐‐ pass around

*Including a review of earlier community‐specific tourism related assessments & studies, and

*Visitor expenditures & niche markets

Program Principles are also found in the Community Tourism Development Manual, Messer (2010)

by the U of M Tourism Center. The manual compiles techniques which have successfully been used in communities across the United States. Copies of the Manual are available online at: https://www.bookstores.umn.edu/viewProduct.cgi?productID=35339#.U8avq0CTEvk


RB

Cut if Limited to 20 Min

New York Mills‐ population of 3,325 (2012) located in West Central Minnesota

20 in original 2007 Tourism Assessment Program

Fertile – population 835 (2012) located in Northwest Minnesota


Lindstrom‐ population 4,408 (2012) located in East Metro Minnesota



LT

We wanted the evaluation to help us improve the program as well help the community in their tourism development. (Should I mention Rani’s feedback from communities that the U takes information but doesn’t give it back to us?)

These were the three methods we used and I will talk about each of them in the next three slides:

1. REM – used to find out what actually happened in the community

2. Community Capitals – used to identify the assets the communities used to make the things happen that they identified in the REMs

3. ORID – We went back to the community to share their REMs and the community capitals. ORID provided a framework to internalize and discuss the results.

We selected the following methods for this evaluation process because we wanted to continue a collaborative approach towards community tourism development that would provide community members 1) insight into the programs purpose and 2) opportunities to take what they learn from the evaluation process to apply in their day to day work.

Hansen Kollock, D.H., Flage, L, Chazdon, S., Paine, N., and Higgins (2012): Identified a need for different mode of program evaluation within Extension that aligns better with human cognition‐more so than experimental research, and outlined the steps that can be taken using Ripple Effect Mapping (REM) to undertake such an qualitative evaluative process.

Coaching workshop participants through this sample coding and ORID processes also allowed educators to contextualize the community capitals framework into event examples provided by participants and also allowed participants the opportunity to frame their development goals using the new development terminology of the community capitals framework.


LT

How many are familiar with REM (read about it, used it)? It is a way to understand intended and unintended results of a program. One change or success can lead to other changes successes which are captured in the REM.

A REM is conducted in the community with program participants as well as others who were not directly involved in the program. In this case, we had

14 in New York Mills

18 in Lindstrom

22 in Fertile

In pairs they did an appreciative inquiry process to identify successes they were proud of regarding tourism development since the TAP program (held 4‐5 years earlier) During this session, Participants were asked questions similar to those Rani posed to you here earlier.

These successes were mapped and linkages between successes (where one thing lead to another) were identified.

We noted a lot of enthusiasm in each session. The process allowed people the time to reflect on what they had accomplished over time – when doing the day to day work, it becomes overwhelming and loose sight of the big picture

9

RB

This is just an example of one map for the community of Lindstrom.

By recording effects as they were mentioned in each community discussion, you can see in this example how an initial effect can be visualized to move outward, into secondary and tertiary effects.

You can also see how (to map out the TAP program “performance”) we identified three main knowledge areas addressed by the TAP program, these included increasing each community’s awareness of : their potential visitor markets, local community infrastructure, and also potential internal and external relationships that could be fostered to accomplish their tourism development goals.


LT

How many are familiar with Community Capitals (fingers 1‐5). Emery, Fey & Flora

Our research team took each item on the Ripple Effects Maps and determined which capitals were needed to make that piece on the map happen. Items could be coded with more than one capital. These were places into an Excel spreadsheet for tabulation

Natural capital refers to those assets that abide in a location, including resources, amenities, and natural beauty. Natural capital might include parks, farm land, and features of the landscape or of nature.

Cultural capital reflects the way people “know the world” and how to act within it. Cultural capital might include ethnic festivals, multi‐lingual populations, or a strong work ethic.

Human capital is understood to include the skills and abilities of people, as well as the ability to access outside resources and bodies of knowledge in order to increase

our understanding and to identify promising practices. Human capital might include a local leadership development program, a high level of educational achievement within the population, or a cluster of skilled crafts people.

Social capital reflects the connections between people and organizations or the social glue that make things happen.

Political capital reflects access to power and power brokers, such as access to a local office of a member of Congress, access to local, county, state, or tribal government officials, or leverage with a regional company.

Financial capital refers to the financial resources available to invest in community capacity building, to underwrite businesses development, to support civic and social entrepreneurship, and to accumulate wealth for future community development. Many times financial capital becomes the focus of community efforts.

Built capital refers to the infrastructure that supports a community such as telecommunications, industrial parks, main streets, water and sewer systems, roads, etc. Built capital is often a focus of community development efforts.


RB

From the ripple maps we identified significant tourism development impacts in the following areas of knowledge for each community:

1) increased community cooperation through relationship building (Relationships),

2) enhanced marketing efforts resulting from a better understanding of visitor markets (Visitor Markets),

3) expanded awareness of the area’s Natural, Cultural, Recreational and Event attractions (Community Infrastructure).

The results listed here were presented back to each of the three communities during the second workshop session in 2013‐2014.

New York Mills

14 participants in 2013 REM mapping session

17 participants in 2013 workshop session

Fertile

22 community participants in 2013 REM mapping session


Lindstrom

18 community participants in 2013 REM mapping session



RB

1. Coding findings aligned with unique interests of each community: New York Mills focused on mobilizing their Cultural Assets, Fertile focused on mobilizing their Financial Assets, and Lindstrom focused on mobilizing their Social Assets.

The percentages you see here add up to more than 100% because many of the effects reported engaged more than community capital. The total number of reported effects in each community that were coded were as follows: New York Mills 71, Fertile 80, and Lindstrom 90.

2. Presenting coded results back to the communities of New York Mills and Fertile helped broaden participants views concerning assets that their communities could allocate towards tourism‐specific development efforts.

3. Darger (2014) also suggests that while REM and coding mapping results within the Community Capitals Framework can help re‐engage and sustain long term development efforts of communities; there is still room for the process to be more refined to meet the needs of federal and possibly even state program evaluation and reporting processes.

Mapped event examples using specific capitals :

New York Mills: local company Lund Boats financially supporting new cultural programs

Fertile: Got funding to finally build a bridge to the Agassiz Environmental Nature Center which opened up additional potential for new lodging expansion and revenue generating opportunities.

Lindstrom: Prior to engaging with the Tourism Assessment program the community did not communicate well with its neighbors. Yet after being engaged with the program (and strengthening their social connections ) they have opted now to refer to themselves as the Chisago Lakes Area of Minnesota.


LT

Once the analysis was complete, we were committed to bringing the research back to the communities.

Lapp (2010) “When we process information and think clearly, we move through four different modes, sometimes very quickly. These modes of thinking can be labeled: Objective, Reflective , Interpretive and Decisional.”

Following the analytical process of ripple effect mapping, we felt that the ORID discussion format was an opportunity to align the summative thinking by participants with their normal

Objective questions focus the participants attention on what is external and observable

Reflective questions focus participant attention on their reactions to external information

Interpretive questions focus participant attention on their valuation/feeling about their thinking

Decisional questions focus the participant attention on the behaviors or actions they would take based on their interpretations.

During the last phase of the community workshops in Fertile and Lindstrom in 2014, ORID process was used to record participants action items that were based on ripple effect maps and the community capitals.


RB

Overarching takeaway : “Engaged programs need engaged evaluation”

Communities value reflecting on their progress towards development goals and the programming efforts that help them accomplish this.

When using the community capitals with stakeholders in programming and evaluation:

• Its more helpful to introduce Community Capitals Framework concepts at launch of a project

• Use of the Community Capitals Framework can help broaden the definition of success within a community by stakeholders


References:

Chazdon, S. (2014). Ripple Effect Mapping: A Tool for Evaluating the Impacts of Complex Interventions. Training Presentation to Michigan State University Extension.

Chazdon, S. and Paine, N. (2014). Evaluating for Public Value: Clarifying the Relationship Between Public Value and Program Evaluation. Journal of Human Sciences and Extension 2(2).

http://www.jhseonline.com/#!current‐issue/co3j

Cousins, J.B. and Whitmore, E. (1998). Framing Participatory Evaluation . New Directions for Evaluation #80. Winter 1998

onlinelibrary.wiley.com/doi/10.1002/ev.1114/

Darger, M. (2014 ). “Capturing the Ripples from Community –Driven Business Retention and Expansion Programs”. Journal of Extension 32(2).

http://www.joe.org/joe/2014april/tt6.php

Emery, M. C. Fey, and C. Flora. (2006). “Using Community Capitals to Develop Assets for Positive Community Change.” CDPractice: Promoting Principles of Good Practice. Community Development Society. Iss. 13.

http://www.ndcompass.org/community‐building/library.php?libItemID=1273#.U7ngdUCTEvk

Hansen Kollock, D., Flage, L., Chazdon, S., Paine, N., and Higgins, L. (2012). Ripple Effect Mapping: A “Radiant” Way to Capture Program Impacts. Journal of Extension 50 (5).

http://www.joe.org/joe/2012october/tt6.php

Lapp, C.(2010). “Focused Conversation ORID Method”. Technology of Participation®. Presentation to LA 8206 University of Minnesota: Making Urban Landscape Space class December 04, 2010

http://ispimi.org/images/meeting/082212/focused_conversation_univ_minnesota.pdf

Martilla, J.A. and James, J.C. (1977). “Importance‐Performance Analysis”, Journal of Marketing, Vol. 41 (1), pp. 77‐79. http://www.jstor.org/stable/1250495?origin=JSTOR‐pdf

Messer, C.C. et al. (2010).Community Tourism Development. 3d ed. University of Minnesota Extension.

http://www.bookstores.umn.edu/viewProduct.cgi?productID=35339#.Uma_jT‐d5Co

Messer, C.C. (2014) Evaluating Developmental Impacts of Community Tourism Educational Programming: A Minnesota Example. IACD Conference Presentation, Glasglow, Scotland.

Ritchie, J.R.B & Crouch, G.I. (2003). “The Competitive Destination: A Sustainable Tourism Perspective ”UK; CAB International: CABI Publishing.

http://books.google.com/books?isbn=085199847X

Robinson, J.W. & Meikle‐Yaw, P.A. (2007) “Building Social Capital and Community Capacity with Signature Projects: A Case Study of Two Diverse Delta Communities”. Journal of Extension 45(2).

http://www.joe.org/joe/2007april/a4.php



Date post:	16-Jul-2015
Category:	Government & Nonprofit
Upload:	community-development-society
View:	167 times
Download:	0 times

Evaluation of Tourist Assessment Program in Minnesota

Government & Nonprofit