2015-16 TitleIIA(3) Improving Teacher Quality

Competitive Grants-Proposal Needs/Evaluation Expectations

Technical Assistance Webinar—Round IIMarch 16, 2015

1

NEEDS DATATeacher Needs Pre-Proposal Survey

& Other Data

Section A of the RFP

2

Needs Data: Category 1 Projects

• Proposals with all new participants must include:• Evidence at least 1 LEA partner meets federal criteria• Teacher Data

• Pre-proposal needs survey among potential teacher participants in high need and other targeted LEAs

• Student Data• Indication of student needs based on available test scores or

other pertinent student data

• Other Relevant Needs Data• Curriculum, instruction, assessment needs can also be included if they

relate to the proposal, but is not required

3

Needs Data: Category 2 Projects

• Proposals with new and returning participants should include:• Evidence at least 1 LEA partner meets federal criteria• Teacher Data

• Pre-proposal needs survey of potential new teachers; most recent survey data from returning teachers

• Student Data• Indication of student needs new and returning teachers based on available

test scores or other pertinent student data

• Research/evaluation data from current project• Briefly describe current project intervention (in existing project) and

value based on evaluation findings• Provide summary of findings to date, from evaluation of current project,

particularly data representing effects on teachers (including comparison group data) and their students

4

• Teacher Participant Needs Survey Data• MDE ITQ needs survey with option to add project-specific questions (use

only the subject matter section that is pertinent to your proposal)• Current needs survey reflects State Board of Education approved standards

at the time of this grant cycle• Available for five subject areas eligible for an ITQ grant:

• English Language Arts (Common Core State Standards)• Mathematics (Common Core State Standards)• Science (Michigan Science Grade Level Content Expectations)• Social Studies (Michigan Social Studies Grade Level Content Expectations)• World Languages (Michigan Standards and Benchmarks)

• May be administered in paper/pencil or electronic form

5

Needs Survey Data: All Projects

• SAMPI can assist in preparing customized paper/pencil or on-line survey

• SAMPI can compile and return raw data for use in proposal/ program development (need 2-3 work-day turnaround time; data provided in Excel or Access)

• Projects must submit a summary of needs survey data (tables, graphs, summary statements, etc.) and any additional data (student test scores, etc.) in the narrative portion of the proposal

• Submit raw data with code numbers in database (databases available for basic needs survey) in appendix

6

Needs Survey Assistance: All Projects

Example From Needs Survey

INSTRUCTION AND ASSESSMENT PROFESSIONAL DEVELOPMENT NEEDS

Need Already

Met

Low Need

(1)

High Need

(5)

Priority?(Check one for each.)

(Check one box for each.) Yes No

A. Using inquiry-based instructional strategies 0 1 2 3 4 5 Y N

B. Incorporating real-world applications into lessons 0 1 2 3 4 5 Y N

C. Learning how to integrate other curricular topics into lessons 0 1 2 3 4 5 Y N

7

PART A. INSTRUCTION AND ASSESSMENT PROFESSIONAL DEVELOPMENT NEEDS Rate each of the following items based on your personal professional development instructional and assessment needs, with 1 = low need; 5 = high need; 0 = need has already been met. Indicate whether the item is a high priority for you (yes or no).

Adding Project-Specific Items

All projects should have all respondents complete all existing items in all sections.

Projects may add specific instruction and assessment needs to Part B if desired. Extra rows are provided for additional items.

Part C is specific to proposed project content; each project should select their appropriate grade level content/subject matter section and delete all other content tables from the survey. Extra rows are supplied at the end of each content table are provided for additional items.

Part D of the needs survey is a section for projects to ask any open-ended questions desired.

• You may delete Part D if you do not have any other questions you would like to ask.

DO NOT REMOVE ANY ITEMS from the tables; if an item is not relevant to your project, simply ignore that data when analyzing needs survey results and planning your proposal.

8

Student Test Score Data

• Obtain Michigan standardized test score results for the LEA partners relevant to the subject area of your project

Directly from the LEAs

From MDE database• Include a table of student test data in the appropriate

section of your proposal narrative

9

Other Sources of Needs Data

10

• Other Data• Provide other data pertinent to partner needs if

available, such as LEA-generated student or teacher data, curriculum/instruction/assessment needs, etc.

• Previous Research/Evaluation Findings• Describe previous actions taken by IHE partner in

comparable projects to address identified needs• Describe value of proposed intervention based on IHE

partner research/evaluation findings

Analyzing and Submitting Needs Data

• Summarize and present all pertinent needs data

• Do not rely on only ONE source of needs data: identify commonalities in needs data across all sources; the same needs identified from different sources strengthens your case for which needs to address

• Review your needs data to determine links and relationships among the various types of available data

• Prioritize identified needs based on their importance and project-level resources/expertise to address them

11

Teacher/Building Commitment Forms

• Informal “Memo of Understanding” for teachers

• Informal “Memo of Understanding” showing partner school principal support

• Reminders:• Participation not duplication of effort in another project• Avoid conflict of interest for teachers or building

12

EVALUATION

13

Evaluation Goals

1) Improve programming through use of evaluation data

2) Determine the impact of project and statewide effort on new and returning participants

3) Identify strengths and limitations of the projects

14

Proposal Section D (Narrative and Scoring Rubric): Evaluation

All sections of the scoring rubric must be clearly and adequately addressed in the narrative of your proposal.

Label each section of your proposal as per the sections of the rubric.

Project-Level Evaluation

• Evaluation strategies:• Common cross-site pre/post surveys• Pre/post lesson observations• Interview sample of participants• Collect teacher artifacts• Gather evidence of impact on students

• Use additional instruments/procedures specific to project as desired to determine impact or gather implementation data

• Gather and report participation/activity data• Support a person with adequate time dedicated to evaluation

• Internal staff or contractual, with appropriate expertise and without major program coordination duties

15

Cross-Site State-Level Evaluation

• Coordinated by SAMPI-Western Michigan University• Advise on local project use of data collection instruments

and procedures• Advise on evaluation as requested• Assist in coordinating cross-site meetings • Conduct observation of sample of project PD activities• Compile data from across projects• Prepare periodic statewide reports• Assist with pre-proposal needs surveys

16

ELEMENTS OF THE EVALUATION

17

Determine Impact on Teachers/Other Participants

• Pre/post surveys• Plus longitudinal analysis for continuing

projects

• Pre/post lesson observations• Plus longitudinal analysis for continuing

projects

• Sample of interviews of participants• Collect and analyze a sample of teacher

artifacts18

Determine Impact on Teachers/Other Participants

• Projects with both new and returning teachers (Category 2) must also report on differences observed• Comparison group analyses of data from

new and returning participants• Analyses may be quantitative, qualitative or

both

19

Determine Impact on Students of Participating Teachers

• Procedures to identify changes in student learning might include:• Pre/post tests• Pre/post surveys• Focus group interviews• Assessment of student work

• Collect sample of student artifacts to show changes

20

Required Participation in Cross-Site Activities

• Project directors and evaluators are expected to participate in cross-site activities throughout the duration of the grant• First six months: 1 face-to-face meeting in

Lansing, 1 webinar• Remainder of project funding period: 2 face-

to-face meetings in Lansing, 2 webinars

21

Required Common Reporting

• Required Common Reporting• Participation levels and demographics, PD types,

overall assessment of progress toward goals• Final evaluation report• End of project participant-specific data consistent

with cross-site statewide data collection• Sharing findings at cross-site sessions

22

End-of-Project Reporting

• Director Report•Use standard report format available in electronic version

•SAMPI can provide Access (or Excel) database to projects to maintain participation records and to facilitate report preparation

23

Part #1 Sample Reporting Tables

Category Number Major Target

Audience (Yes or No)

1. Number of different teachers served by the project

2. Number of different administrators served by the project

3. Number of different paraprofessionals served by the project

4. Number of different parents served by the project

5. Number of “others” served by the project

24

Part #2: Director Perceptionsof Project Accomplishments

• Rate progress toward outcomes• Provide evaluation/other evidence to

support rating

25

Part #2: Director Perceptions of Project Accomplishments-Sample Form

Section A:#1: Intended Outcome (Type in Outcome Statement):

Rating of Progress Toward Outcome: Rate the degree of progress you believe was made in accomplishing this outcome on a 5-point scale, with 1 = no progress and 5 = fully accomplished.

1 2 3 4 5

Evidence to Support Rating (Type in rational for your rating or, if pertinent, note that there is direct evidence in evaluation report section-see below):

Discussion of Progress in Evaluation Section

Yes No Page No.

26

Complete the following tables, one for each of your intended outcomes as per your proposal. Outcomes are statements of intended impacts or results that will occur as a result of your professional development programming or other project interventions. Prepare this section in collaboration with project evaluator.

Part #3: Nature of Professional Development/Interventions

• Hours, participants, schedule by type of intervention

• Role of college faculty/content experts• Problems planning, implementing project• Problems recruiting teachers

27

Part #3 :Nature of Professional Development/Interventions

Sample Form

PD Formats

Number of Hours of this PD

Provided Over Entire Project

PD Participants(Key: T=Teachers, A=Administrators,

PP=Para-Professionals, P=Parents, O=Others)

When was PD conducted?

(Key: RSD=Regular School Day, AS=After School, EVE=Evening, SAT=Saturday, SUM=Summer)

Workshops (usually one- or half-day sessions)

Institutes (5 or more days usually in summer)

College Course Work (for credit)

E-Learning Courses (self paced web-based course)

28

Part 3: Nature of PD/InterventionsSample Form

NamePosition/

InstitutionPrimary Role

Hours of Involvement

29

3. Use the chart below to describe higher education faculty (both content and education faculty) or other external expert.

PARTICIPATION IN YOUR PROJECT.

Part #4: Evaluation Report Required Components

Prepared by evaluator in collaboration with project director• All projects:

• Data collection• Progress toward project outcomes• Lesson observation data• Teacher and student artifacts• Impacts on participating students• Effectiveness of project partnership• Other

• Category #2 Projects• Results of comparison studies

30

Questions?

• SAMPI contact information:• Crystal Stein or Tessa Souers

crystal.stein@wmich.edu

tessa.souers@wmich.edu

• Mark Jennessmark.jenness@wmich.edu

• Tel: (269) 387-3791

SAMPI is located at Western Michigan University in Kalamazoo, MI

31