Formative Evaluation as Community Development

Daniela StehlikDaniela StehlikAlcoa Research Centre for Stronger CommunitiesAlcoa Research Centre for Stronger Communities

Curtin University of TechnologyCurtin University of Technology

Lesley ChenowethLesley ChenowethSchool of Social Work & Applied Human SciencesSchool of Social Work & Applied Human Sciences

The University of QueenslandThe University of Queensland

Outline of presentation

• Key Learnings• Program Context• Formative Evaluation Philosophy• Evaluation Dimensions• Methods adopted • Strengths & Challenges

Key Learnings -Building Communities through

Evaluation

• Spatial communities• Practice communities• Human service communities

Program Context• Rural/remote• People with disabilities and their

families• Government staffed and funded• Pilots• Quest for better service delivery

models

Formative Evaluation Philosophy Framework

• Social Justice• Participatory• Action research principles• Ethical practice

Formative Evaluation Philosophy Framework - how?• Evaluation as staff development• Through Stakeholder Reference

Group• Involved in professional

development activities• Confidentiality/anonymity

Evaluation dimensions

• Complexities• Multi-level interventions• Cultural change agenda• Searching for ‘best practice’• Pressure to get evidence ‘out’• Rural/remote• Spectrum of disability service

availability

What were the indicators used?

• From Program goals– At the level of …

What were the indicators used?

• From Program goals– At the level of …

• Government Policy• Dept’l Program

management• Field Practice• People with a

disability and their families

• Community capacity building

What were the indicators used?

• From Program goals– At the level of …

• Government Policy (4)• Dept’l Program

management (3)• Field Practice (6)• People with a disability

and their families (5)• Community capacity

building (7)

22 levels ….

Methods adopted ….

• Focus groups• Questionnaires• Indepth interviews• Measurement of social relationships• Participant observation• Client records analysis• Practioner as researcher

Evaluation dimensions

• Complexities• Multi-level interventionsMulti-level interventions• Cultural change agenda• Searching for ‘best practice’• Pressure to get evidence ‘out’• Rural/remote• Spectrum of disability service availability

Multi-level Interventions

Community

FamiliesIndividuals

Multi-level Interventions

Community

FamiliesIndividuals

Program addressed each level and the interactions between

Multi-level Interventions

Community

FamiliesIndividuals

Program addressed each level and the interactions between

Multi-level Interventions

Community

FamiliesIndividuals

Program addressed each level and the interactions between

Multi-level Interventions

Community

FamiliesIndividuals

Program addressed each level and the interactions between

Multi-level Interventions

Community

FamiliesIndividuals

Program addressed each level and the interactions between

Key Learnings -Building Communities through

Evaluation• Spatial communities

– Worker/families/ngos/govt etc.

• Practice communitiesPractice communities– Workers/ngos/govt/academeWorkers/ngos/govt/academe

• Human service communities– govt./ngos/academe

University

Field

Research site in field

Building the Practice Community

New knowledge informs training

of new practitioners

New practitionersimplement new

knowledge

Researcher-practitionersexchange >>

new knowledge

CD in practice

• The role of the practitioner• Capacity building doesn’t just

‘happen’• Facilitation & Intervention are

requiredBuilding a 21st century model of human service practice

Strengths of Evaluation Approach

• Accommodated complexity• Provided both quantifiable and

illustrative evidence• Gave value beyond the evaluation

per se• Enabled participation including

people with a disability

Strengths cont.

• Built practitioner confidence • Enhanced program profile within

dept.• Worked in longitudinal and short term

situations • Enabled timely feedback of findings • Aided writing of report

Challenges

• Assumed high level evaluator knowledge of program, practice & issues

• Political sensitivity• Distance across sites limited

participant observation • Demand for face to face contact • Problematic access to client databases

& records

Thank - you