Early Childhood Outcomes Center1 Involving Families.

Early Childhood Outcomes CenterEarly Childhood Outcomes Center 1

Involving Involving FamiliesFamilies


Informing FamiliesInforming Families

What are you doing to inform families about the data collection? Why it is occurring What it involves What it means for them and their child


www.the-eco-center.orgwww.the-eco-center.org

Federal and State Activities

State Activities

http://www.fpg.unc.edu/~eco/whatstates.cfm#State_sites



Preparing FamiliesPreparing Families

Helping families be active participants in the discussion What is working? What is not working?

General principle: Families need to know what to expect


What Do We Expect from What Do We Expect from FamiliesFamilies

Yes - That they will be able to provide rich information about their child’s functioning across settings and situation

Maybe but not necessarily – That they will know whether their child is showing age appropriate behavior


Involving Families in a Involving Families in a Conversation about Their ChildConversation about Their Child

Avoid jargon Avoid questions that can be answered

with a yes or no “Does Anthony finger feed himself?”

Ask questions that allow parents to tell you what they have seen “Tell me about how Anthony eats”


Involving Families in a Involving Families in a Conversation about Their ChildConversation about Their Child

What is working? What is not working?


Involving Families in the Involving Families in the Rating DiscussionRating Discussion

What % of families are participating?

How are you doing this? What is working? What is not working?


Families’ Right to COSF Families’ Right to COSF InformationInformation

All families have a right to know what ratings have given to their child -- and to the records containing the information.


Quality Review of DiscussionQuality Review of Discussion

ActivityActivity

Including Families in the Rating Discussion


Looking for Quality DataLooking for Quality Data

I know it is in here somewhere


ActivityActivity What factors work to improve the

quality of your data? What factors work to lessen the

quality of your data? How to address these factors?


Take Home MessageTake Home Message

If you conclude the data are not (yet) valid, they cannot be used for program effectiveness, program improvement or anything else.

What do you if the data are not as good as they should be?

Answer: Continue to improve data collection through ongoing quality assurance


Many steps to ensuring quality dataMany steps to ensuring quality data

BeforeGood data collection/Training

Good data system and data entry procedures

During

Ongoing supervision of implementation

Feedback to implementers

Refresher training

AfterReview of COSF records

Data analyses for validity checks





During



Refresher training




Promoting quality dataPromoting quality data

Training and support before and during data collection

Analysis of the data after data collection

Data system and verification after data collection


Promoting Quality DataPromoting Quality Data

Through training and communication

related to: Assessment Understanding the

COSF process Age expectations Data entry



Through training materials, such as Video team and child examples Written child examples “Quizzes” for ensuring learning

Refresher trainings –

Beware of Drift!!



Through data systems and verification, such as

Data system error checks Good data entry procedures





During



Refresher training




Ongoing supervisionOngoing supervision

Review of the process Is the process high quality? Are teams reaching the correct rating?

Methods Observation Videos


Quality Review of COSF Quality Review of COSF Team DiscussionTeam Discussion

1. Do all team members participate in the discussion?

2. Is parent input considered in the rating? Give examples.

3. Is the team documenting the rating discussion? Give examples.

4. Does the team discuss multiple assessment sources? What are they?



5. Does the team describe the child’s functioning, rather than just test scores? Give examples.

6. Does the discussion include the child’s full range of functioning, including skills and behaviors that are age appropriate, immediate foundational, and leading to immediate foundational? Give examples.


Quality review through process Quality review through process checkschecks

Provider surveys Self assessment of competence Knowledge checks Process descriptions (who participates?) Identification of barriers

Kansas survey Alaska’s survey

Questions from Alaska’s SurveyQuestions from Alaska’s Survey

3. How would you rate your own level of proficiency with the COSF process? (please select only one)

I am confident I know how to do it, and I do it well

I know how to do it, but I need some more practice and assistance

I understand it to a point, but I need more training

I do not know how to do this yet

4. Some cases are different from others, but of the choices below, which process seems to be the most typical in your experience? (please select only one)

I gather information and determine COSF ratings on my own

I gather information and consult with another provider to determine COSF ratings

I gather information and consult with the family to determine COSF ratings

I gather information, discuss it with a team and the team determines the COSF ratings


Ongoing SupervisionOngoing Supervision

Feedback to teams is critical Refresher training Beware of

Auto pilot Drift



ActivityActivity

Observe team video

Evaluate quality





During



Refresher training




Quality Review of Completed Quality Review of Completed COSFsCOSFs

Is the COSF complete?

Is there adequate evidence for the basis for the rating?

Does the evidence match the appropriate outcome area?

Is the evidence based on functional behaviors?


Quality Review of Completed Quality Review of Completed COSFsCOSFs

Is there evidence that the child’s functioning across settings and situations considered?

Are the ratings consistent with the evidence?


Quality Review of COSFQuality Review of COSF

ActivityActivity

Review completed COSF with errors


Promoting quality data through Promoting quality data through data analysisdata analysis

Examine the data for inconsistencies

If/when you find something strange, look for other data that might help explain it.

Is the variation caused by something other than bad data?


The validity of your data is The validity of your data is questionable if…questionable if…

The overall pattern in the data looks ‘strange’ Compared to what you expect Compared to other data Compared to similar

states/regions/school districts


COSF Ratings - FallCOSF Ratings - Fall

Rating Class 1 Class 2 Class 3 Class 4

1 3 1 1 2

2 4 1 2 2

3 5 2 3 3

4 6 3 2 4

5 1 4 5 4

6 1 5 5 4

7 0 4 2 1

Spring

Fall 1 2 3 4 5 6 7 total

1 1 4 2 7

2 1 1 5 6 9 3 1 26

3 2 15 14 27 19 6 83

4 4 4 21 39 28 12 108

5 1 12 14 71 86 48 232

6 1 3 21 48 63 136

7 2 18 23 56 99

Review Total 2 13 38 60 185 207 186 691

Outcome 3: Appropriate Action

Spring

Fall 1 2 3 4 5 6 7 total

1 1 4 2 7

2 1 1 5 6 9 3 1 26

3 2 15 14 27 19 6 83

4 4 4 21 39 28 12 108

5 1 12 14 71 86 48 232

6 1 3 21 48 63 136

7 2 18 23 56 99

Review Total 2 13 38 60 185 207 186 691


Spring

Fall 1 2 3 4 5 6 7 total

1 1 4 2 7

2 1 1 5 6 9 3 1 26

3 2 15 14 27 19 6 83

4 4 4 21 39 28 12 108

5 1 12 14 71 86 48 232

6 1 3 21 48 63 136

7 2 18 23 56 99

Review Total 2 13 38 60 185 207 186 691


Spring

Fall 1 2 3 4 5 6 7 total

1 1 4 2 7

2 1 1 5 6 9 3 1 26

3 2 15 14 27 19 6 83

4 4 4 21 39 28 12 108

5 1 12 14 71 86 48 232

6 1 3 21 48 63 136

7 2 18 23 56 99

Review Total 2 13 38 60 185 207 186 691



OSEP CategoriesOSEP Categories

OSEP CategoriesClass 1

(%)Class 2

(%)Class 3

(%)

e. Maintained Age Appro Trajectory 23 16 24

d. Changed Traj – Age Appro 15 23 13c. Changed Traj – Closer to

Age Appropriate 32 34 37

b. Same Trajectory -Progress 28 21 25

a. Flat Trajectory – No Prog. 2 6 1


Questions to askQuestions to ask

Do the data make sense? Am I surprised? Do I believe the data?

Believe some of the data? All of the data?

If the data are reasonable (or when they become reasonable), what might they tell us?


ValidityValidity

Validity refers to the use of the information Does evidence and theory support the

interpretation of the data for the proposed use?

Or Are you justified in reaching the inference

you are reaching based on the data? Standards for Educational and Psychological Testing (1999) by

American Educational Research Association, American Psychological Association, National Council on Measurement in Education


The validity of your data is The validity of your data is questionable if:questionable if:

?


….not all providers are not knowledgeable about in the COSF process

…not all providers are careful with the COSF process

…the data look “strange” …etc.





During



Refresher training



Date post:	27-Mar-2015
Category:	Documents
Upload:	anthony-lyons
View:	216 times
Download:	0 times

Early Childhood Outcomes Center1 Involving Families.

Documents