Communities for Children evidence-based program requirement – where

to from here?Elly Robinson

FRSA Conference, 10-12 November 2015Brisbane, QLD

Communities for Children Facilitating Partners - objectives

To improve the health and wellbeing of families and the development of young children (0-12 years), paying special attention to: Healthy young families Supporting families and parents Early learning School transition and engagement

The evidence-based program requirement

New grant agreements From 1 July 2015 at least 30% of funding used

for direct service delivery should be used to purchase high quality, evidence-based programs as identified by the Expert Panel.

Increases to 50% from 1 July 2017

Dual approach

The “guidebook” Evidence-based programs that meet a rigorous

set of criteria “Other” programs

Recognise that good programs are already being implemented

A slightly less rigorous set of criteria

The “Guidebook”

Criteria for inclusion Objectives of program are in line with CfC FP model. Targets children aged 0-12 years and their families. Documented information on program is available:

Aims, objectives and theoretical basis; Program logic or similar; Target group for program; and Activities of the program and why they are important.

The “Guidebook”

Criteria for inclusion of programs Training manual or documentation that allows for replication in

Australia is available. A good quality evaluation with at least 20 participants. Evaluation shows positive outcomes (and no negative effects

reported). Program has been replicated (or has potential for replication).

30 profiles now available

The “Guidebook” – recent additions

Healthy & Safe A home‐based education resource designed to

equip parents with learning difficulties with the knowledge and skills necessary for managing home dangers, accidents and childhood illness.

Parent-Child Mother Goose A group program for parents/carers and their

babies/young children. Supports the development of secure parent-child attachment, promotes children’s speech development and enhances families’ community inclusion.

https://www.sd79.bc.ca/schools-programs/programs/early-learning/parent-child-mother-

goose/

The “Guidebook” – recent additions

Smalltalk A service enhancement that can be embedded within

community-based supported playgroups. Designed for participating parents experiencing vulnerable circumstances with children aged 6-months to 4 years.

Through the Looking Glass A therapeutic parenting program based on attachment

theory principles which aims to strengthen attachment relationships between children aged 0- 5 years and their parents/carers in cases where attachment has been compromised.

Assessment of “other” programs

Criteria for inclusion - must be documented A theoretical and/or research background to the program. A program logic (or theory of change, or logic model). Activities in the program which generally match good practice

in meeting the needs of the target group. An evaluation (with at least 20 participants) has established

that the program has positive benefits for the target group. Staff members are qualified and/or trained to run the program.

Needs to tell the “story” of your program http://bit.ly/1MLW411

3. Provisional assessment

In recognition of when more time is needed to meet criteria for 30% requirement.

Must be able to: Articulate research and/or theoretical background of the

programme; show that some form of evaluation has been conducted in the

past (may not meet the rigorous evaluation criteria yet); and outline your plan for meeting the criteria in the online guide by 30

June 2016.

Assessment of programs - summary

Wide variation of quality and documented effectiveness

Total programs so far: 13 programs fully approved 42 provisionally approved 7 not approved

https://aifs.gov.au/cfca/expert-panel-project/list-programmes-submitted-approval-cfca

Can we deliver programs on approved list?

Fully approved programs meet a minimum standard for the purposes of the 30/50% requirement.

Not evidence-based as per the “guidebook” programs – would still prefer these were used.

If FPs wish to explore further, we can provide contact details for the program developers.

Provisionally assessed programs – suggest waiting.

Same program – different locations

In some cases, the same program is being run in multiple locations

Program fidelity = staying true to the original program design

Evidence/evaluation relates to particular program

To what extent is the alternative version different?

Program adaptation

Acceptable Changing language, translations Replacing cultural references Adding relevant, evidence based materials to make the program more

appealing to potential participantsRisky/unacceptable

Reducing number/length of sessions (“dosage”) Removing topics Changing the theoretical approach Lowering the level of participant engagement

Building the evidence base

Intention is to continue to build our understanding of “what works” - with your help

“Nudge” programs along the evidence-base continuum Avoid recreating the wheel where possible. Three things over next two years:

Move provisionally approved programs to fully approved. More fully approved programs into guidebook. Focus on 50% requirement, including increasing the pool of

available programs.

Theory/research basis for programs - feedback

Importance Does something already exist that is proven to work, or can reliably

inform our work? What is already known about addressing the needs of our target

group?Many variations on how addressed. Best examples:

Used target group needs as basis Explored what was known already about addressing these needs Used this to frame a response Clear pathway to activities and outcomes

Theory/research basis for programs - feedback

Examples Well-known programs that have no or limited

theoretical/research background, “good practice” activities, evidence of effectiveness (often commercial products)

Longstanding programs with limited/dated research basis Programs developed using target group input & practitioner

knowledge Drawing from different evidence “pots”– the playgroup

example

Other criteria

Program logic Varied quality and generally indicative

of quality of program Program pathway needs to make sense

– “if-then”. The “roadmap” for the program

Activities Need to match “good practice” - e.g.

group vs individual service, who delivers?

Often a lack of detail provided.

Evaluation

Good examples Reasonable sample size to draw conclusions Some level of rigour and appropriate methodology Use of validated tools Not just “feedback sheets” – demonstrated,

measured change from start to end of program Different approaches – degree of discretion for the

purposes of the requirement

How can we help?

CFCA inquiry helpdesk Provides assistance with enquiries related to the

latest stats and facts, evaluation advice, research findings, practice/policy pointers or many other topics.

To access the research and information helpdesk call 03 9214 7888 or use our contact form at https://aifs.gov.au/cfca/contact

Industry List

Helps to connect FaC service providers with research, practice and evaluation experts.

“Screened” members of Industry List: Quality services, fixed price, help to draft proposal.

Guide to use: https://aifs.gov.au/cfca/expert-panel-project/

information-service-providers/guide-using-industry-listOr contact us to talk!

Overall reflections

Terrific opportunity to work with sector – thank you!Main feedback

Need a coherent pathway through program (what is the program’s “story”?)

Most difficult seemed to be longstanding programs Some commercial products have a weak evidence base –

be careful Use the Industry List – they have a track record

Important points for provisionally approved programs

Purpose of project: Build capacity of organisations to deliver quality, evidence-based

programs that work. Other programs, innovations – outside of the 30-50%

All documentation must be submitted by 30 June 2016. Make sure plans reflect this.

Due to volume, please submit complete documentation to CFCA.

Contact us

Expert Panel project https://aifs.gov.au/cfca/expert-panel-project

Industry list and CfC queries fac-expert-panel@aifs.gov.au

Phone us 03 9214 7888 Elly, Sharnee or Kat