Deconstructing Standard 2b

Sharon Valente

Savannah College of Art and Design

1

svalente@scad.edu912-525-5873May 14, 2012

2b: Data Collection, Analysis, and Evaluation

• Regular and systematic data collection, analysis, and evaluation for the purposes of improving candidate performance, program quality and unit operations

• Multiple assessments from internal and external sources at transition points

• Data are summarized, analyzed, and reported by discipline; and are disaggregated for alternate route, off-campus, and distance learning programs

2

Key Criteria for Meeting the Expectations of Element 2b

• System uses multiple assessments from internal and external sources

• System is maintained and data are collected regularly

• Data are regularly and systematically– Compiled - Aggregated– Summarized - Analyzed

3

• Maintained using information technology

• Disaggregated by discipline and for alternate route, off-campus, and distance learning programs

• Records of formal complaints are maintains and resolutions are documented

4

Key Criteria for Meeting the Expectations of Element 2b

Sub-elements of Standard 2b (1)

The professional education unit maintains an assessment system that provides regular and comprehensive information on applicant qualifications, candidate proficiencies, competence of graduates, professional education unit operations, and preparation program quality.

• 1 AFI cited

5

Sub-elements of Standard 2b (2)

Using multiple assessments from internal and external sources, the professional education unit collects data from applicants, candidates, recent graduates, faculty, and other members of the professional community.

• 0 AFI’s cited!!!!

6

Sub-elements of Standard 2b (3)

Candidate assessment data are regularly and systematically collected, compiled, aggregated, summarized, and analyzed to improve candidate performance, preparation program quality, and professional education unit operations.

• 15 AFI’s cited

7

Sub-elements of Standard 2b (4)

The professional education unit disaggregates candidate assessment data when candidates are in alternate route, off-campus, and distance learning programs.

• 0 AFI’s cited!!!!

8

Sub-elements of Standard 2b (5)

The professional education unit maintains records of formal candidate complaints and documentation of their resolution.

• 5 AFI’s cited

9

Sub-elements of Standard 2b (6)

The professional education unit maintains its assessment system through the use of information technologies appropriate to the size of the professional education unit and institution/agency.

• 9 AFI’s cited

10

Scenarios

• DISCLAIMER: The information presented does not represent any one institution, nor does this purport to be the ideal format and discussion for 2b. Any errors found are those of the authors, and should not be attributed to the Georgia Professional Standards Commission (GaPSC).

• The items in blue and/or italics represent hypertext links to further data. The reader may assume that these reports provide the suggested data. For purposes of the present exercise, the reader may assume that similar data is presented for all teacher preparation programs at the given institution.

11

Scenario One

• Assessment DescriptionThis assessment consists of a cumulative G.P.A. in 12 required courses that address content knowledge for Physical Education teacher candidates. Teacher candidates must maintain a 3.0 (out of a 4.0) cumulative G.P.A., with no grades below a C- to remain in the program and to enter practicum and clinical practice. The candidates’ major adviser records and tracks these requirements throughout the program.

The scoring guides developed with the department faculty define the course letter grades from A to F. Separate guides were developed for foundation courses and pedagogy and content courses.

• Data AnalysisAlong with the guides, the mean G.P.A. data for all 12 required courses is presented. The overall G.P.A. mean score of 3.35 is above the required departmental mean score of 3.0. However, six candidates did score lower. These candidates were required to retake those courses.

12

Scenario One

• EvaluationIt is apparent that our teacher candidates score higher in the Pedagogical/Content courses (mean of 3.48) as compared to the mean for Foundation courses of 3.08. Many candidates excel and are more comfortable with the “activity” courses. Also, there is an inherent difference in learning domains in the two areas: Foundation courses focus on the cognitive domain while the pedagogy/content courses also include the psychomotor and affective domains.

In an effort to address this discrepancy, instructors will continue to meet students individually or in groups to provide additional instruction when needed.

Unacceptable, Acceptable, or Target?

13

Scenario Two

• Assessment Description

The Clinical Practice Teacher Candidate Performance Evaluation (CPTCPE) comprises two rubrics: The Clinical Practice Teacher Candidate Performance Evaluation Rubric and the NAPSE Addendum. The university supervisor uses the CPTCPE to provide assessment feedback to the candidates at least four times over the course of each quarter. The candidate’s performance is formally assessed at midterm (end of first placement) and at the end of their clinical practice (end of second placement), immediately prior to program completion. At those times, based on observations and consultation with clinical teachers, university supervisors complete an evaluation instrument for each teacher candidate. Candidates must demonstrate or provide evidence to the university supervisor and cooperating teacher that they are able to meet or exceed expectations on all 52 elements of the final CPTCPE to pass Clinical Practice.

14

Scenario Two

• Data Analysis

The data was collected from the Spring 2009 cohort of teacher candidates in Clinical Practice. The data indicate a slight improvement trend from midterm (1.25) to final (1.28). The lowest mean scores were for 5.1 (Creates a learning community in which individual differences are respected) and 6.1 (Applies knowledge of students’ abilities/disabilities to positively impact student learning), although all candidates did score ”Acceptable.”

15

Scenario Two• Evaluation

The overall change in mean scores from midterm to final is small but positive, which seems to indicate growth over time. It is surmised that adjustments to the new NAPSE Addendum may have caused some of the discrepancies and negative trends in the data.

To address this issue, we have embarked on a process of rewriting our Clinical Practice Primer for supervisors. The revised document will be available for the onsite visit.

At present, clinical practice supervisors do not grade candidates’ impact papers, their reflections on their teaching, student learning, and making decisions based on those reflections. This appears to have affected candidates’ scores in this assessment. In the future, impact papers will be part of this assessment.

Unacceptable, Acceptable, or Target? 16

Scenario Three

• Student Complaint Policy

The institution has established policies and procedures for resolving student complaints. These policies are outlined in the Student Complaint Policy in the Student Handbook and the Catalog. Students are encouraged to first attempt to resolve certain complaints informally by contacting the office responsible for the area relevant to the complaint. For example, students are referred to the residence hall director for housing complaints; to the professor or academic chair for academic complaints; and to the bursar's office for financial concerns. The institution ensures that its policies are published each academic year in the Student Handbook and the Catalog. The catalog is mailed to all applicants and an updated copy is sent to all incoming students. It is also published at the public website. Thus, the policies are disseminated to the students.

17

Scenario Three

Additional Complaint Mechanisms

• Student Ombudsman

Office of the Student Ombudsman exists solely to serve the students. The student ombudsman is a designated neutral party who does not advocate any particular point of view. As an impartial complaint-handler, the student ombudsman strives to see that people are treated fairly and equitably. A student may consult with the student ombudsman on a variety of matters and concerns, including but not limited to, academics, judicial concerns, and unethical behavior.

 

18

Scenario Three

• Ensuring Policies are Followed

To ensure that the published policies are followed when resolving student complaints, all written student non-academic complaints, other than those concerning FERPA and/or ADA, are routed through the Office of Student Success. The assistant to the dean and vice president for student success maintains a log of student complaints and the corresponding outcomes. This log will be available for the onsite visit.

Unacceptable, Acceptable, or Target?

19

Rubric for Evaluating Evidence

20

Unacceptable Acceptable Target

The unit does not use multiple assessments from internal and external sources to collect data on applicant qualifications, candidate proficiencies, graduates, unit operations, and program quality.

Using multiple assessments from internal and external sources, the unit collects data from applicants, candidates, recent graduates, faculty, and other members of the professional community.

Assessment data from candidates, graduates, faculty, and other members of the professional community are based on multiple assessments from both internal and external sources that are systematically collected as candidates progress through programs.

Rubric for Evaluating Evidence

21

Unacceptable Acceptable Target

The unit does not regularly and comprehensively gather, aggregate, summarize, and analyze assessment and evaluation information on the unit's operations, its programs, or candidates.

The unit maintains an assessment system that provides regular and comprehensive information on applicant qualifications, candidate proficiencies, competence of graduates, unit operations, and program quality.

The unit's assessment system provides regular and comprehensive data on program quality, unit operations, and candidate performance at each stage of its programs, extending into the first years of completers' practice.

Rubric for Evaluating Evidence

22

Unacceptable Acceptable Target

The unit cannot disaggregate candidate assessment data when candidates are in alternate route, off-campus, and distance learning programs.

The unit disaggregates candidate assessment data when candidates are in alternate route, off-campus, and distance learning programs.

These data are disaggregated by program when candidates are in alternate route, off-campus, and distance learning programs.

The unit does not maintain a record of formal candidate complaints or document the resolution of complaints.

The unit maintains records of formal candidate complaints and documentation of their resolution.

The unit has a system for effectively maintaining records of formal candidate complaints and their resolution. The unit is developing and testing different information technologies to improve its assessment system.

Rubric for Evaluating Evidence

23

Unacceptable Acceptable Target

When candidate assessment data are collected, the data is not compiled, aggregated, summarized, or analyzed.

Candidate assessment data are regularly and systematically collected, compiled, aggregated, summarized, and analyzed to improve candidate performance, program quality, and unit operations.

These data are regularly and systematically compiled, aggregated, summarized, analyzed, and reported publicly for the purpose of improving candidate performance, program quality, and unit operations.

The technology used to support the unit’s assessment system is used sporadically.

The unit maintains its assessment system through the use of information technologies appropriate to the size of the unit and institution.

The unit is developing and testing different information technologies to improve its assessment system.

Thank You!

24