Post on 04-Aug-2020
transcript
Loyola Marymount UniversityProgram Evaluation
Project REAL
Retaining highly qualified teachers who provide Evidence-based instruction,
Access to general education curriculum, &
Leadership in special education advocacy
LMU Special Education Program
• Small program with 2 full time faculty & 1 clinical faculty– Dr. Victoria Graf, Program Director– Marianne Mitchell, Program Assistant Director – Dr. Tisa Aceves, Project REAL PI– Hasmig Balian, Project Assistant
• Candidates take courses outside of our program & department
Project REAL Objectives
Objective 1: To restructure & redesign existing course content & field experiences to integrate competencies reflecting EB instructional & assessment practices
Objective 2: To help ensure candidates acquire the necessary skills to provide leadership in special education advocacy
Project REAL Objectives
Objective 3: To prepare candidates to provide appropriate access to the general education curriculum for all learners
Objective 4: To provide professional development for general & special education faculty
Objective 5: To develop & maintain an effective, rigorous mentoring & induction program that increases candidates’ successful completion of the program while developing their professional expertise & commitment to the profession
Major Changes to Program
1. Complete course review & development to reflect EB practices & new CA standards
2. Implement developmental sequence of courses3. Coordinate courses across departments4. Develop website to support candidates5. Shift away from evaluation of candidates to
mentoring of candidates6. Provide peer mentors 7. Infuse content reflecting Universal Design for Learning8. Infuse content for Special Education Advocacy
Program Evaluation Measures
• Special Education Program Survey• Observation Template• Revised Course Syllabi Evaluation• Signature Assignments (Livetext)• Workshop evaluations• Needs assessment (faculty, instructors, field
supervisors)• Course Evaluations• University Supervisor Evaluation by Candidates
Special Education Program Survey
• Drafted & administered survey November / December 2008 via Survey Monkey
• Surveyed completed by: – Alumni– Candidates– Supervisors– Instructors – Employers
(N=200)
SPED Program Survey - Sample Items1 = Sufficient attention is provided2 = Some attention needed3 = Moderately more attention needed4 = Significantly more attention needed
_____ 1. Developing the IEP and participating in the IEP process_____ 2. Classroom organization and management (routines, schedules, etc.)_____ 3. Whole-group behavior management (rules, consequences, etc.)_____ 4. Providing appropriate interventions for severe behavior problems_____ 5. Lesson planning and delivery_____ 6. Differentiation and individualization of instruction_____ 7. Evidence-based Instructional methodology_____ 8. Addressing the needs of English Learners with disabilities_____ 9. Providing culturally and linguistically appropriate instruction _____ 10. Conducting and interpreting assessment
Part II Qualitative Questions
SPED Program Survey
• Identified areas of strength & weakness
• Helped confirm annual exit interview comments from candidates
• Contributed to course changes
RelativeStrengths
Relative Weaknesses
Behaviormanagement
Lesson planning
Conducting & interpreting assessments
Differentiation
Understanding special education law
Interventions for severe behavior problems
Developing collaborativerelationships
Adapting curriculum
Observation Template• Observation template reflects the evaluation
of candidates in the field in the areas of Instruction, IEPs/Collaboration, Assessment, Professional Behavior (formative & summative tool)
• Contributors included: program faculty, faculty from Dept. of Clinical Ed., external evaluator, university supervisors
• Measure will be piloted with candidates this spring
Observation Template (sample items)C. Instruction 0 1 2 3 4
Ensures that students are engaged in learning activities. Comment:
0 = Not observed / insufficient evidence
1 = Does not meet expectations / practice not consistent with standard
2 = Is beginning to meet expectations/developing practice
3 = Meets expectations /maturing practice
4 = Is highly effective in meeting expectations / practice that exemplifies standard
Communicates the learning objective, purpose of the lesson, and learning expectations. Comment:
Communicates directions & instructional procedures of lesson in multiple ways.Comment:
Provides multiple opportunities & time for students to practice & apply what they’ve learned.Comment:
Draws on students’ background, interests & dev. learning needs for instructional planningComment:
Uses EB instructional techniques that reflect an awareness of different student learning styles. Comment:
Differentiates lesson using EB instructional strategies that minimizes barriers & expands learning opportunities for all learners (e.g., assistive technology).
Comment:
Demonstrates knowledge & understanding of the appropriate use of technology for information collection, analysis, & management in the instructional setting.
Comment:
Creates environment that removes potential instructional barriers thereby maximizing access to content.
Comment:
B. Classroom Environment/Behavior 1. Provides, creates and maintains positive, supportive classroom climate that
is safe and conducive to learning.Comment:2. Establishes, and maintains consistent standards of classroom behavior.
Addresses all behavior issues in a consistent manner (classroom level).Comment:
3. Manages individual student behavior effectively and in a positive, constructive manner (individual student level).Comment:
4. Establishes efficient classroom routines. Examples include transitions, beginning lessons, and general classroom routines.Comment:
Observation Template (sample items)
Revised Course Syllabi Evaluation• Program alignment of 325T project/program
objectives with new CA standards & LMU Conceptual Framework
• Program evaluation of previous program against new CA standards (0-2 point system)
• Determine EB practices through review of literature
• Advisory Board evaluation of new program syllabi– Evaluation Part I (special education courses) to be
completed mid January
– Evaluation Part II (courses outside of department) to be completed beginning of February
Revised Course Syllabi Evaluation
• Training for Advisory Board Course Syllabi Evaluators– Powerpoint Overview with voice over
– Sample Syllabi & completed sample rubric with voice over explanations
– CD with readings & resources
– Copy of Cover Letter, 325T Abstract, new CA Standards, & new program course sequence
Course Evaluation Rubric
CTC Standards, Project REAL Objectives (REAL) & SOE Conceptual Framework (CF), Program Objectives (PO)
Mark an X under the following ratings (0-2) indicating how much evidence the course demonstrates a particular standard.
AWARENESS / KNOWLEDGE (Passive Reception of Information)
SKILL DEVELOPMENT/USE (Analysis / Synthesis)
APPLICATION IN FIELD
0 0.5 1 1.5 2Final Score
No evidence that the standard is included in the class syllabus
Standard is stated in class syllabus (e.g., course objectives, course description)
Standard is stated in syllabus and includes 1 or more of the following: • tests or quizzes of knowledge/awareness• Professor demonstration • Discussion •Lecture/PPT •Reading •Observation (Passive) •Media Use
Standard is stated in syllabus. Course includes readings, tests (skill demonstration), & assignments. Course ALSO includes 1 or more of the following possible projects demonstrating application of standard:• Lesson Plans• Student Activity during class • Journal Response • Case Study (analysis of data) • Simulations
Standard is stated in the syllabus. Course includes required readings, tests (skill demonstration)-projects-assignments. Course ALSO includes application of standard in the field (classroom or school):• Teaching a lesson• Conducting assessment • Participation in IEP • Case Study (assessment or instruction) • Conducting FBA
Write the final point value for each standard rated. (0-2)
S2:Pr/Eth Practice (REAL 2, CF 3)
Process for Completing Rubric
S2: Pr/Eth Practice (REAL 2, CF 3)
S3: Div.Learners (CF 1)
S4: Collaboration (REAL 2, CF 3)
S5: Assessment (REAL 1, )
S6:Technology (REAL 3)
S7:Transition
Process for Completing RubricAWARENESS / KNOWLEDGE
(Passive Reception of Information)0 0.5 1
No evidence that the standard is included in the class syllabus
Standard is stated in class syllabus (e.g., course objectives, course description)
Standard is stated in syllabus and includes 1 or more of the following: • tests or quizzes of knowledge/awareness• Professor demonstration • Discussion •Lecture/PPT •Reading •Observation (Passive) •Media Use
Process for Completing Rubric
SKILL DEVELOPMENT/USE (Analysis / Synthesis) APPLICATION IN FIELD
1.5 2Standard is stated in syllabus. Course
includes readings, tests (skill demonstration), & assignments. Course ALSO includes 1 or more of the following possible projects demonstrating application of standard:• Lesson Plans• Student Activity during class • Journal Response • Case Study (analysis of data) • Simulations
Standard is stated in the syllabus. Course includes required readings, tests (skill demonstration)-projects-assignments. Course ALSO includes application of standard in the field (classroom or school):• Teaching a lesson• Conducting assessment • Participation in IEP • Case Study (assessment or instruction) • Conducting FBA
X
Process for Completing Rubric
Final ScoreWrite the final point value for each standard rated. (0-2)
Rating: 2
Next Steps in Course/Program Evaluation
• Conduct Advisory Board evaluation
• Conduct External Evaluators evaluation– Leonard Baca, Ph.D. (Univ. of Colorado, Boulder)
– Norma Lopez Reyna, Ph.D. (University of Illinois, Chicago)
• Continue collaboration with other departments
• Submit syllabi for 325T Evaluation in April
Question for 325Ts
• How can we better connect program & candidate evaluation to K-12 student achievement?– Current forms of evaluation
• Signature assignments include evaluations & interventions of diverse students in the field
• 2-3 courses use AIMSWEB to assess & monitor student progress