Closing the loop in Evaluation of safety

in remote NT communities

using an ethical, robust and participatory

approach

Department of Families, Housing, Community Services and Indigenous Affairs

NTER whole-of-government evaluation

Evaluation Principles:• ‘person centred’ approach• Light footprint• Long term assessment of

change• Learning partnership• Systems focus

Local Research Projects

• Provides a deeper understanding of the context where data is being collected

• Train and mentor local Indigenous people in research skills

• Develop and strengthen existing local research knowledge, skills and competencies

• Develop a pool of skilled participatory researchers (participatory panel)

Components of the NTER evaluation

Secondary

data analysis

Community

impact

studies –

Community Safety

and Wellbeing

Research Study

Monitoring Reports and

analysis

Evaluation of ' bundled measures’

NTER Revie

w

NTER Redesig

n

WoG Evaluation

Report

Evaluation of separate

measures

Other

intelligence

Top Down

NTE

R W

hole

of G

over

nmen

t

Eval

uatio

n re

port

GAPS

Bottom up

Service Provider Survey

Community Safety & Wellbeing Research Study

Coordination&

Engagement Study

Program evaluations Program admin data

Evaluation Design

Community Safety and Wellbeing Research Study

• Provided a voice for Indigenous Australians affected by the NTER

• Explored how the health, welfare and safety initiatives as part of the NTER impacted on

community members lives.

FaHCSIA team

• Kim Grey – psychology and sociology, mixed methods background

• Sue Sutton - ABS and survey experience, quantitative data collection and analysis

• Nathalie Baxter – International Development background including Participatory Rural Appraisal (PRA) techniques

Methodology

• Research was designed to be both consistent and flexible• Structured quantitative survey• Participatory approaches using methods that

suit individual communities

• Allowed triangulation of monitoring and evaluation data with the views of people directly affected by the NTER

Survey Questionnaire• Original survey developed by Australian Institute of

Criminology (AIC)

• Questions comparable with other research e.g. National Aboriginal and Torres Strait Islander Social Survey (NATSISS)

• Pretested by Experienced Aboriginal Researchers

• Piloted in one community

• Modified questions and language

• Rolled out in 16 other NTER communities

Communities

• 17 communities chosen to cover a cross section of NTER communities – size, location, police presence

• 1,323 people participated in the quantitative survey

• A stratified quota sample of community members aged 15 and over were surveyed based on the sex and age profiles of each community from the 2006

Census of Population and Housing.

Research Consultants

• Four consultants engaged through departmental panel of consultants with expertise in participatory research methods

• Consultants had prior relationships with communities and participatory research experience

• Most consultants had experienced Aboriginal researchers as part of the team

Consultancy team

• Bowchung Consulting

• Colmar Brunton Social Research

• N-Carta Group

• Dr Sithole and ARPNet

Participatory research cycle

Training local researchers

Ownership of report by community

Consent

Capturing ‘community voice’

Giving report back

Employment opportunities e.g. ABS census

Community builds on research e.g. community safety planning

Ethics Board

Consent

• Research did not go ahead until there was signed consent from each community

• Differed from community to community

• Appropriate time should be allocated in all projects to do this right

Training local researchers

• Consultants engaged and trained local researchers

• Over 60 Indigenous researchers were employed

• Around 10 experienced Indigenous Researchers

• 55 local researchers from the participating communities

Capturing the community voice – Quantitative Survey

Local Researchers played a number of roles: • Project leaders • Interpreters• ‘Brokers’ to encourage people to participate• Conducted surveys and interviews• Organised groups and conducted

participatory research• Entered data

Capturing the community voice – participatory qualitative research

• Ranking of changes and challenges

• Different methods were used to achieve the same outcome

• Changes and challenges were identified from the responses/comments received while conducting the survey

• And/or from discussion groups

• Ranking was conducted by individuals or groups

Difficulties working in remote communities

Issues that researchers had to work around included:• Sorry business• Court dates• Meetings in communities• Ceremonies• Sporting events• Gambling• Pay days/Grog Days• School holidays – leaving community• Apathy about being over-researched• Mistrust of Government

Difficulties working in remote communities

• Difficulty in getting to communities• Cheeky dogs• Persistent rain / heatEngaging local researchers• Local researchers hard to attract if they have

other income sources e.g. mining royalties• Lack of understanding of importance of

research• Feel they don’t have the right to ask people

questions• Low literacy levels within the community

Researchers working in their own communities presents challenges

• Working across difficult family/community relationships

• Getting caught up in family/community related issues

• Researchers knowing something about the respondent

• Living situation can impact on energy levels of researchers

• Home environment (project material gets lost or is destroyed) and

• Issues of “jealousing” from members of the community

Reporting back to communities

• Results of the survey and participatory comments written up in a ‘Community Analysis Report’

• The report is jointly owned by FaHCSIA and the community

• Community can use the report for future planning

Handing report back

• All consultants went back to communities to present the report to them

• Differed from community to community• Formal meetings with local reference group• Community BBQ• Meetings with major stakeholders• Separate presentations with groups e.g.

women’s group, men’s group etc. • Walk through the community and discuss

with community members

“Getting Balanda people from outside might feel hard to express our feelings to them, countryman feel comfortable and relaxed with our own people”

Comment from community member about using Local Researchers….

• “It’s good to have u mob come talk, its people’s point of view, how they feel before and now, yes I do, yes I did and it help us understand what’s bad and what’s good”

Comment from community member about the research…

Comment from research team member…

“As we were doing our work some people were interested in working with us and getting involved. They said it was good we were doing the work”

Feedback from Communities

"One of the best research projects ever done for us...the way the guys went about it, the way the guys engaged with the community and Yalu and got them involved and the timeliness of reporting back."

Thank you

kim.grey@fahcsia.gov.au