Moving Beyond“Can Do It, Can’t Do It”

Margaret HeritageCindy Bagwell

Amy ScrinziDenise Nelson

Innovative Approaches to the Assessment of Students in Grades K-3

CCSSO National Conference on Student Assessment

June 22, 2015 | San Diego, CA

Overview

• Assessment Perspective

• Project Overview

• Challenges & Lessons Learned

• Validation Study

• Questions & Answers

Assessment Perspective

Assessment: Two Views of the Learner

Past-to-Present:

Retrospective

Heritage, 2013

Formative Assessment

• Assessment for Learning (Black & Wiliam, 1998; Black, Harrison, Lee, Marshall & Wiliam, 2003; Gipps, 1994)

• Proximate to Learning (Erickson, 2007)

• Assessment in the flow of activity and interactions in the classroom (Heritage & Heritage, 2014; Swaffield, 2011)

Learning as Progression

Assessment to Support Learning

• Where am I going?

• Where am I now?

• Where to Next?

Black & Wiliam, 1998, Hattie & Timperley, 2007; Harrison & Howard, 2009; Sadler, 1989

Wherever you are is perfect…

Wherever you are is perfect… for now.

Project Overview

What Research Tells Us

• Educational researchers• Economists• Neuroscientists• Developmentalists• Molecular biologists• Genomic scientists

• The PK-3rd grade years are the most promising window of opportunity during which to influence children’s lifelong trajectories.

Kauerz, 2013

Race to the Top-Early Learning Challenge Grant

NC State Law “Read to Achieve”

NC’s K-3 Assessment Vision

13

5 Domains of Learning and Development

K-3 Formative Assessment

Formative AssessmentA process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to help students improve their achievement of intended instructional outcomes.

AERA/APA/NCME, 2014

CCSSO, 2006

What is essential?

How do we measure

this?

How do we implement for sustainability?

What is working? What could be?

Focus Groups & Input Sessions

External Reviewers

KEA Pilot

State-levelAdvisory Group

Kindergarten Teacher Survey

NC K-3 Formative Assessment Process

What is essential?

NC K-3 Assessment Think Tank

Proposes claims, or learning goals, for each domain

Emphasizes a focus on the whole child

Promotes the use of a formative assessment process

How do we measure

this?

Early Childhood Educators

Early Childhood Educators

Content ExpertsContent Experts

SpecialistsSpecialists

TeachersTeachers

Catherine Scott-LittleUNC Greensboro

Catherine Scott-LittleUNC Greensboro

Margaret HeritageCRESST/WestEd

Margaret HeritageCRESST/WestEd

NC K-3 Formative Assessment Process:Kindergarten

Domain Constructs

Approaches to Learning Engagement in Self-Selected Activities

Cognitive Development Object Counting

Emotional-Social Development Emotional Literacy

Health & Physical DevelopmentFine Motor DevelopmentMidline Motor Development

Language Development & Communication

Following DirectionsLetter Naming Book Orientation & Print Awareness

NC K-3 Assessment Design Teamw/Dr. Margaret Heritage

Construct ProgressionsDescribes how students’ learning of important concepts and skills

develops over a period of time.

SKILLS: Identify the competencies within each “understanding”, ranging from simple to more complex

SKILLS: Identify the competencies within each “understanding”, ranging from simple to more complex

PERFORMANCE DESCRIPTORS: Paint a picture of performance, providing examples of how students could demonstrate their understanding

or skill at each stage of the progression

PERFORMANCE DESCRIPTORS: Paint a picture of performance, providing examples of how students could demonstrate their understanding

or skill at each stage of the progression

UNDERSTANDINGS: Identify the major concepts within a particular construct

UNDERSTANDINGS: Identify the major concepts within a particular construct

Assessment Means

NC K-3 Formative Assessment

• Situations– Intentionally planned instructional activities

designed to give teachers guidance on or examples for how to set up a learning situation to learn about students through observation & probing.

General Specific

General Situation

• Select Learning Targets

• Identify Opportunities for Eliciting Evidences of Learning/Probing

• Interpret the Evidence

• Adapt/Respond to Learning Needs

Specific Situation

• Select Learning Targets• Preparation• General Description• Elicit Evidences of

Learning/Probing• Interpret the Evidence• Adapt/Respond to Learning

needs

Family Contributions

• Use of Family Questions to support home-school partnerships– Face-to-face conversations

• Home visit• Conference

– Phone calls

Family Questions

•What is your child most excited about learning?

•How does your child typically approach new things, such as meeting new people or going to new places? How do you help your child prepare for new experiences?

•What new things would you like your child to learn? Why are these things important to you?

Family Questions, continued

•How does your child show emotions (e.g., happiness, sadness, surprise, frustration)? How do you respond to each emotion?

•What does your child like to do at home or with family and friends (e.g., favorite games, books, toys, activities)?

•What can we do to help your child be successful?

Development Process

Challenges & Lessons Learned

Your mission…should you decide to accept it…

Write a Construct

Progression

What can the child MAKE, SAY, DO or

WRITE?

Be aware of GRAIN SIZE

Include: UNDERSTANDINGS, SKILLS, PERFORMANCE

DESCRIPTORS and SITUATIONS

Formative Assessment

Process

HOW NC Approached Writing Construct Progressions…

• What does the NC Think Tank Report state?

• What does research say?

• What do we know from personal and professional expertise about how children learn?

• What do content specialists say?

HOW NC approached writing Construct Progressions…

• And then…• We begin to write…and write…and write…• Are we getting the grain-size right?• Gather input from the field• Discussions and “meeting of the minds”• Edit, edit, edit…revise, revise…edit, edit,

edit…revise, revise…edit, edit, edit…

Formative Assessment Process …finding the balance….

• The Assessment Design Team had to

–Work under rigorous timelines while considering other commitments

–Manage the work of multiple constructs and groups

–Know when to stand firm and when to compromise

Lessons Learned…What We Know Now…

• Feedback is necessary– varied stakeholders, many perspectives

• Response to feedback is key– positive comments of appreciation from pilot

schools

• Being cutting edge is both rewarding & challenging– not for the faint of heart but requires heart

Lessons Learned…What We Know Now…

• We did it!– we know we are moving in the right direction – it’s

what is best for children

WHEREVER WE ARE IS PERFECT …

FOR NOW!

Validation Study

Purpose of the NC Kindergarten Pilot

• Provide feedback on…

– Professional development

– Assessment content and format

– Electronic platform

– How assessment worked in practice

NC Kindergarten Pilot

• UNC-Charlotte Research Activities:

– Teacher Survey

– Classroom Observations

– Teacher and Administrator Interviews

– Examination of Evidences

NC Kindergarten Pilot

• Who?

– 248 Kindergarten Teachers

– 5,000 Kindergarten Students

• Where?

– 81 Schools across the state

• When?

– 1st 60 days of school

NC Kindergarten Pilot Findings

Commonalities in Classrooms Successfully Implementing KEA

Small class size: averaged 14 students

Students independently transitioned from one classroom activity to another.

Teachers used self-created implementation resources to assist with documentation.

Commonalities in Classrooms Successfully Implementing KEA

Formative assessment practices were imbedded into all classroom activities, so KEA data collection fit into teachers’ previously established routines.

“I want to warn you that you’re not going to see some prepared ‘something’ today; you’re just going to see

what I do everyday.”~Pilot Teacher

Commonalities in Classrooms Struggling to Implement KEA

• Large class size: averaged 22 students

• Students struggled to transition independently between classroom activities.

• Classrooms were located in schools/districts without a strong history of formative assessment practices.

Commonalities in Classrooms Struggling to Implement KEA

• Teachers approached KEA implementation as they would a summative assessment:• Created additional activities to “test” each

child’s ability individually or as a group.

NC KEA ContentNC KEA Content

• Developmental Appropriateness:– 71% of survey respondents felt the content was

developmentally appropriate for kindergarten.

“This really validates what we do and deal with everyday…there’s so much that needs to happen before you see a lot of academic changes. These young children are going to be growing socially tremendously [in the beginning of the year] and administrators need to understand [teachers]

have all this other stuff to get in place before they can start moving academically.” ~ Pilot Teacher

NC KEA ContentNC KEA Content

• Some participants felt that while the content was developmentally appropriate, the KEA was still not suited for current kindergarten classrooms:

“Is this developmentally appropriate? Yes, but to be honest we don’t have the ‘freedom’ to use it. We are

mandated by so many other expectations for our children that there is no way to do the KEA the way it should be done and still be responsible for the content we must

teach and then assess them on (state mandated summative tests by the way).” ~Pilot Teacher

NC KEA ContentNC KEA Content

• Some participants felt that while the content was developmentally appropriate, the KEA was still not suited for current kindergarten classrooms:

“If this was 5 years ago this would have been perfect, but kindergarten is looking more and more like first

grade. So while this IS developmentally appropriate, unfortunately it’s now more suited for PreK because kids are expected to enter Kindergarten with most of

these skills.” ~Pilot Teacher

• Teachers… See the value of a formative process Love the whole child focus Appreciate support for motor and emotional-

social constructs Have become more reflective

• Teachers…• Are overwhelmed • Need a lot of professional development & support• Struggle to see the connection b/t the assessment

content & what they do with students• Worry about how the information will be used

NC Pilot Implications

• Professional Development– Emphasize critical components of the formative

assessment process – Provide real world examples of formative

assessment in action– Include support staff (e.g., teacher assistants,

specialists)– Devote adequate time and intensity with

technology platform– Incorporate implementation resources

NC Pilot Implications

• Supportive Conditions– Strong background in formative assessment – Existence of PLCs focused on data-driven

instruction– Implementation teams at the state, region,

district, and school levels– Administrators with experience in early

childhood

Questions?

Contact Information

Margaret HeritageCRESST/WestED

mheritag@ucla.edu

Cindy BagwellNC DPI Project Manager

cindy.bagwell@dpi.nc.gov

Amy ScrinziNC DPI Project Lead

amy.scrinzi@dpi.nc.gov

Denise NelsonNC Assessment Design TeamNC Regional Implementation Team

deniseasplus@gmail.com

tiny.cc/NCK3FAPtiny.cc/nck3fap_educator