MMD SummitEvaluation Update and conversation primer6/13/17

OverviewConversation will be segmented into three major buckets:

Data acquired through:1. End of event survey2. Next steps document during cadre time3. Spider web/radar plots

2

3

End of event survey

Survey IntroductionAs a reminder, we pulled a lot of the information that we normally collected through survey out and into cadre working documents. As such, we’ll start with our qualitative questions around:

1. Anxiety going forward2. Focus going forward3. End goal4. How to evaluate progress towards that end goal5. How the participant hopes to contribute

And then we have both qual and quant data around HQPD

4

Anxiety going forwardWe asked folks: “Please share with us any hesitation or anxiety that you might have, even small, now that you understand more about the structure and processes inherent in the MMD relationship.”

5

Results showed:

6

Area N

Concern about DESE communication/guidance 16

Around specific processes 8

Local infrastructure challenges 15

Hesitation around data 11

Lack of understanding of MMD 14

Not enough time 10

Lack of teacher buy-in 14

Extra work 14

No anxiety 12

Concern about DESE communication/guidanceHere is an example:

“I worry about rushing this process with districts. Too often well-intentioned processes get rushed and then lead to "going through the motions" without authentic practice. Please do not rush this. This summit already showed signs of this as we spent more time listening to rationale than we did getting to work in our teams and grapple with how we will implement the process.”

7

Around specific processesHere is an example:

“I worry that there is more to this than we have been told. It seems that there are things that have not been shared with us. In my experience with DESE the end result is rarely what we were told to expect.”

8

Hesitation around dataHere is an example:

“The data collection platform appears cumbersome, and could be much more efficient. Especially in this digital age.”

9

Lack of understanding of MMDHere is an example:

“It does not seem to have clear goals in mind, I am still a bit confused about how this is different from the CW grant.”

10

Also:Shared as a DESE concern specific to policy:“Application of accountability, over support, down the road.”

Shared as a DESE concern about there being something under the covers:“This event was possibly one of the most poorly organized, put -together-at-the-last-minute things I have ever encountered in my career. It's becoming a rather obvious that several non-CW groups are simply the control group for a research project, and it's interesting to see how this might play out.”

11

More specificsWithin local infrastructure challenges (15), most frequently cited was a lack of alignment to the current district practices (6), as well as competing initiatives (2) and no plan going forward (2) mentioned by more than one person

Within hesitation around data (11), more than half were specific to the virtual learning platform (6).

12

We asked folks: “As a step towards these goals or targets, please share one or more of your highest priorities that you would like support with through your Missouri Model District participation during the 2017-2018 school year.”

Focus going forward

13

Results showed:

14

Area N

School Improvement 67

Infrastructure 48*

Academic 18

Staff development 8

Leadership 3

Buy-in 2

Staff 1

*Included: Building teams/collaboration (21), coaching (13), alignment, identification of needs, consistency, and implementation levers

We asked folks: “Please share your “end-zone” goals or target that you would like to achieve following participation as a Missouri Model District?”

End goal

15

Results showed:

16

Area N

School Improvement 18

Academic/RtI/Instruction 50*

Culture/Climate 2

Collaboration 23

Intra-district 20

Inter-district 1

Alignment: State policy and resources 4

*Included: DBDM (15), ACL (7), CFA (6) and others

Academic/RtI/InstructionHere is an example:

“As a building leader I recognize the need to continue to change the climate and culture of my building to become a safe, productive learning environment that makes learning fun and contagious. We will have a primary focus on the ACL Process and the RTI process.”

And….

“Teachers learning how to transfer more responsibility for learning to each student.”

17

DBDMHere is an example:

“For the data team process to become "how we do things" instead of another checklist item.”

18

Collaboration: Intra-districtHere is an example:

“I would like to see this program help us where we have a deficit (Coaching, leadership, PD) and have an outside source come in and let us know how we're doing on the parts we already have set in place.”

19

Alignment: State policy and resourcesHere is an example:

“I want to see a statewide focus of allowing or providing resources to school for better coaching and instructional leadership.”

20

We asked folks: “How could one begin to evaluate progress towards those outcomes? (those stated as end goals)”

How to evaluate progress towards that end goal

21

We asked folks: “What do we need to do to contribute to our team's journey with MMD? (as related to the team’s focus)”

How the participant hopes to contribute

22

23

High Quality PD

General ReactionFolks were less likely to agree that specific desired practices were present, as compared to past project events, as well as compared to other project events that we’ve evaluated. Here is the difference this event as compared to the mean for the last two Shared Learning events.

24

-19%

-21%

-5%

-30%

-7%

-10%

-24%

advanceresources

research-based

explainedconcepts

adequateengagement

expressedopinion

opportunitytoreflect

resourcesformastery

From Qualitative Data

25

Area N

General Gripes 18

Specific to lack of engagement 13

Suggestions 16

Benefits 6

Ongoing needs 8

Communication 3

Kudos 3

General Gripes

Here is an example:

“The approach to material seemed somewhat scattered. Presenters were often in disagreement when answering questions. We spent too much time bouncing around the material with little clarification.”

26

Here is an example:

“While the information was relevant for student and school success I felt the conference could have been condensed into two days by excluding much of the first day's presenters. It was not a positive experience for me to sit from 8:00-4:00 listening to presenters.”

Lack of Engagement

27

Suggestions for event frameworkHere is an example:

“It seemed that the information shared on Thursday would better have been shared on Wednesday, and vice versa. It was almost as if they were soliciting "buy in" to a program before explaining the program.”

And…

“I thought that there was a lot of ambiguity in the initial presentations from DESE rep.s. I realize that they were laying the groundwork for us but I spent a lot of time wondering where they were headed. That was the general consensus of the people ispoke with.”

28

Benefits: specific to alignmentHere is an example:

“Once those were clear on the 2nd day, anxiety was lifted because we realized this goes well with and enhances things we are doing. It seems this will help us continue to move forward and improve.”

29

Communication: Unknown requirementsHere is an example:

“Many participants were confused and distracted by the platform as they didn't understand the process and focused on the platform and its pitfalls without realizing that the process was focused on the 6 areas and not on the platform. Adult learners have the same needs as students; we need clear communication with more hands on learning. Teach us the way you want us to teach our students. Model quality teaching practices authentically. Even though there were opportunities for interaction and discussion, it was much too limited.”

30

31

Knowledge Change

During the event we asked folks to consistently track how they were doing on a number of different event objectives on this scale:

1. Just beginning2. Getting better3. Bullseye

Closer to the Bullseye

32

Bullseye Getting better Just beginningFinal Number 59 280 48Final Percent 15% 72% 12%Percentage Point Change 12% 20% -32%

33

Aspiration

Calculations

Of the 82 total participants for this item, 59 agreed that they aspired to implement what they had learned, and 16 even strongly agreed. The resulting rate was:

72%

34

35

Next Steps

Not Quite Finished

36

37

Spiderwebs/Radar Plots

ANALYSIS“I loved having several breakout sessions to choose from that were differentiated to different levels or interests.”

*Participants prompted about general PD**The survey collapsed DBDM and collaborative teaming as scales due to similarity

38

Coaching

Common Formative

Assessment

Data-based Decision Making

Collaborative Teaming

Effective Teaching and Learning

Practices Leadership

Mean 2.0 2.7 2.9 2.9 2.6 3.0

Mode 1.0 3.0 3.0 2.0 3.0 3.0

SD 0.8 0.6 0.7 0.9 0.6 0.6

From Survey 4.0* 4.4 4.3** 4.3** 3.7 4.2