+ All Categories
Home > Technology > Evaluating ISE (2012)

Evaluating ISE (2012)

Date post: 10-Jul-2015
Category:
Upload: kim-arcand
View: 292 times
Download: 0 times
Share this document with a friend
12
TRAILER Kim Arcand LOGIC MODEL Sketching out how the program will work and devising a plan to measure the outcomes (integrative framework for analysis) EVALUATION METHODOLOGIES FOCUS GROUP, WORKSHOPS Organized discussion with a small group to understand the how’s & why’s. Requires mediation, gives social response and perspective. INTERVIEWS, QUESTIONNAIRES & OBSERVATIONS Asking what visitors think or watching how they interact: building your baseline of qualitative and quantitative data. CREATIVE OR PARTICIPATORY Capturing open, qualitative info by participants drawing, photographing or writing responses. Involving participants in creating the eval. framework Institution or Program Perspective Participant Perspective
Transcript
Page 1: Evaluating ISE (2012)

TRAILER

Kim Arcand

LOGIC MODEL

Sketching out

how the program

will work and

devising a plan to

measure the

outcomes

(integrative

framework for

analysis)

EVALUATION METHODOLOGIES

FOCUS GROUP,

WORKSHOPS

Organized

discussion with a

small group to

understand the

how’s & why’s.

Requires

mediation, gives

social response

and perspective.

INTERVIEWS,

QUESTIONNAIRES &

OBSERVATIONS

Asking what visitors

think or watching how

they interact: building

your baseline of

qualitative and

quantitative data.

CREATIVE OR

PARTICIPATORY

Capturing open,

qualitative info by

participants drawing,

photographing or

writing responses.

Involving

participants in

creating the eval.

framework

Institution or Program

Perspective

Participant Perspective

Page 2: Evaluating ISE (2012)

TRAILER

Kim Arcand

BASIC LOGIC MODEL

Adapted from University of Idaho: http://www.uiweb.uidaho.edu/extension/LogicModel.pdf

INPUTS OUTPUTS OUTCOMES

What is invested•Time•Money•Partners•Equipment•Facilities

What is done•Exhibits•Workshops•Publications•Etc.

Short-termChange in:•Knowledge•Skills•Attitude•Awareness•Motivation

Long-termChange in:•Environment•Social conditions•Economic cond.•Political cond.•Etc.

Evaluation Study: Measurement of process indicators – Measurement of outcome indicators

AUDIENCES

Who•Internal•External

(Formative, summative and anything in between)

Page 3: Evaluating ISE (2012)

TRAILER

Kim Arcand

SPECIFIC LOGIC MODEL

• Plan programs

• Examine feasibility of implementation & potential success

• Create tools to communicate within & outside

• Facilitate evaluation questions

Page 4: Evaluating ISE (2012)

TRAILER

Kim Arcand

From Earth to the Universe (FETTU) evaluation strategy

FINDING THE RIGHT QUESTIONS…AFTERWARDS?

- Based on NSF ISE evaluation rec., "Framework for Evaluating Impacts of Informal

Science Education Projects" (2008)

- Used observation, survey and interview protocols for viewing audiences, as well as staff

of host-partner institutions and local community partners

- Found out good stuff…but had more questions than answers

http://www.fromearthtotheuniverse.org/

Page 5: Evaluating ISE (2012)

TRAILER

Kim Arcand

From Earth to the Solar System (FETTSS), new exhibit in 2011-2013

SECOND TIME AROUND

- Testing findings from FETTU

- ~80 locations worldwide (From cafes in New Zealand to malls in Canada)

- Audience targeted is the non-science expert. Offer multiple access points. Family-

friendly language/set up; free, public, everyday settings; Spanish translations;

Braille/tactile materials.

http://fettss.arc.nasa.gov/

Page 6: Evaluating ISE (2012)

TRAILER

Kim Arcand

2ND CHANCES: FETTSS EVALUATION STRATEGY

Observer: Date: FETTSS Venue: Time of Day:

Composition of Visitor Group:________________ Key: M = adult male F = adult female

G = teen girl B = teen boy g = girl b = boy

Total Visit Time:

Notes: List any site specific notes (eg., the most visited images were

by the restroom or the entry way, etc.) or general observations here:

Behaviors Observed Read Label

Read Aloud Point at Image feature Talk (about exhibit) Show/Explain to others

Ask Questions Other (describe) Visitor Comments Overheard:

Observation sheet:

Page 7: Evaluating ISE (2012)

TRAILER

Kim Arcand

2ND CHANCES: FETTSS EVALUATION STRATEGY

Survey sheet: (exploring casual vs. intentional; science “identity” & expertise level, demographics, etc.)

Page 8: Evaluating ISE (2012)

TRAILER

Kim Arcand

EVALUATING ON A BUDGET: POSTCARDS

What to watch out forErrors: sample size (n=10+), random selection, etcOver-generalizing, SubjectivityEvaluation does not = Research

Page 9: Evaluating ISE (2012)

TRAILER

Kim Arcand

FOCUS GROUPS

Aesthetics & Astronomy is a research project that

studies the perception of multi-wavelength astronomical

imagery and the effects of the scientific and artistic choices

in processing astronomical data.

Runs series of 2-3 focus groups for the qualitative data to

accompany an online survey and in-person interviews for

quantitative data.

Research questions include:

• How much do variations in presentation of color, explanation, and

scale affect comprehension of astronomical images?

• What are the differences between various populations (experts,

novices, students) in terms of what they learn from the images?

• What misconceptions do the non-experts have about astronomy

and the images they are exposed to?

• Does presentation have an effect on the participant?

http://astroart.cfa.harvard.edu

Page 10: Evaluating ISE (2012)
Page 11: Evaluating ISE (2012)

G292.0+1.820,000 light-years

Where do the calcium in our bones and the oxygen in our lungs come from? From supernovae, the exploding stars that create beautiful “supernova remnants” such as this one. This X-ray image of the supernova remnant G292.0+1.8 shows a rapidly expanding shell of gas containing elements such as oxygen, neon, magnesium silicon and calcium that were created both during the lifetime of the star and in the explosion itself. Explosions like this dispersed elements that were necessary to form our Sun and Solar System.

G292.0+1.820,000 light-years

The oxygen we breathe, the iron in our blood, and the calcium in our bones comes in part from exploded stars such as the Cassiopeia A supernova remnant seen here. Most of the elements throughout the Universe, other than hydrogen and helium, were forged in the cores of stars. These elements were then dispersed into space when the stars exploded, later to be used as the building blocks when new stars and planets formed. Our Sun and Solar System—including Earth—contain the materials for life thanks to such previous generations of stars.

G292.0+1.820,000 light-years

G292.0+1.8 is a young supernova remnant located in our galaxy. This deep Chandra image shows a spectacularly detailed, rapidly expanding shell of gas that is 36 light years across and contains large amounts of oxygen, neon, magnesium, silicon and sulfur. Astronomers believe that this supernova remnant, one of only three in the Milky Way known to be rich in oxygen, was formed by the collapse and explosion of a massive star. Supernovas are of great interest because they are a primary source of the heavy elements believed to be necessary to form planets and life.

Page 12: Evaluating ISE (2012)

TRAILER

Kim Arcand

What questions do you want to ask?

What are the outputs?

Who are your audiences?

What outcomes do you want to measure (short term, long term)?

What outcomes are useful to your funding organization(s)?

EVALUATING: CLASS ARCHIVES PROJECT

INPUTS OUTPUTS OUTCOMESAUDIENCES


Recommended