Post on 23-Feb-2016
description
transcript
2013-2014 TELPAS Holistic TrainingAssessment, Research & Evaluation Department
Disclaimer
This training does NOT take the place of reading the appropriate manuals.
2
3
Agenda1. General Information
2. Test Security and Confidentiality Requirements
3. General Information About Holistically Rated TELPAS Assessments
4. TELPAS Additional Roles
5. Prepare to Rate
Holistic Rater TrainingJanuary 28th, 2014
4 General information
5
TELPAS ChangesNew standards were set for TELPAS readingStandards were adjusted to meet the new definition of grade level-appropriate that accompanies the increased rigor of STAAR
Domain weights have shifted in composite score calculations.
Reading is now 50% Writing is 30% Listening and Speaking are each weighted 10%
6
TELPAS ChangesYears in US Schools Data Collection Changes Beginning with calculations made and reported in 2014, students must be enrolled for 60 consecutive school days in a school year for that year to count as a year in US schools.
More specific information regarding these calculations is available on the TEA website
7
TELPAS Assessment Components
Grades K-1Holistically rated observational assessments Listening Speaking Reading Writing
Grades 2-12Multiple choice online tests in grades 2-12 Reading
Holistically rated Writing collections
Holistically rated observational assessments Listening Speaking
8
Eligibility Requirements All ELL’s in grades Kinder – 12th are required to participate All ELL’s in grades Kinder – 12th whose parents have declined bilingual/ESL program services
In rare circumstances, it may be necessary for the ARD committee in conjunction with the LPAC to determine that an ELL receiving special education services should not be assessed in reading/writing/listening/speaking for reasons associate with the student’s particular disability. Participation must be considered on a domain-by-domain basis and reasons must be supported and documented in the IEP and the PRC/SPF
9
Eligibility RequirementsAn ELL from another school district, state, or country who enrolls on or after the first day of the TELPAS testing window will not be assed by the receiving district in the HOLISTICALLY rated domains
Newly enrolled ELL’s in grades 2-12 are required to take the TELPAS reading test; however, TELPAS coordinators must verify with DTC that the student hasn’t already completed testing elsewhere
10Test Security and Confidentiality Requirements
11
Test SecurityMaintaining the confidentiality of the TELPAS program involves protecting the contents of all online assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines
Personnel meet the requirements, have been trained and have signed the appropriate oath to have access to the assessment materials and information
Campuses implement the controls necessary to ensure proper storage of secure materials throughout the test administration
Campuses place confidential documentation (rating rosters) in a limited-access locked storage
12
Confidentiality RequirementsTesting personnel undergo training and sign the oath affirming they understand their obligations concerning the security and confidentiality of the TELPAS program
Testing personnel administer the tests in strict accordance with the instructions
Testing personnel do not view, reveal or discuss the contents of an assessment before, during, or after a test administration
13
Penalties for Prohibited Conduct
Placement of restrictions on the issuance, renewal or holding of a Texas educator certificate
Issuance of an inscribed or non-inscribed reprimand
Suspension of a Texas educator certificate for a set term
Revocation or cancellation of a Texas educator certificate without opportunity for reapplication for a set term or permanently
Any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, as well as any person who fails to report such violation is subject to the following penalties:
14
Penalties for Prohibited Conduct (continued) Release or disclosure of confidential test content could result in criminal prosecution under TEC §39.0303, Section 552.352 of the Texas Government Code, and Section 37.10 of the Texas Penal Code. Further, 19 TAC §249.15 stipulates that the State Board for Educator Certification may take any of the above actions based on satisfactory evidence that an educator has failed to cooperate with TEA in an investigation.
Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments.
15
Security Oaths and Confidentiality Statements All personnel who participate in state –mandated testing or handle secure test materials must meet the eligibility requirements and sign a security oath
Any person who has more than one testing role (e.g. a Rater who also serves as a Test Administrator) must receive appropriate training and sign a security oath for EACH role
Each oath for TELPAS raters, writing collection verifiers and test administrators must be read and completed AFTER the training and BEFORE handling or viewing any secure materials or confidential information
Oaths must be kept on file for at least five years
16
Testing IrregularitiesSerious Sharing answers to the online
calibration activities or practice rating activities in the online training courses
Falsifying TELPAS holistic ratings or writing samples
Directly or indirectly assisting students with responses
Tampering with student responses
Procedural Eligibility Error IEP Implementation Issues Improper Accounting for Secure
Materials Monitoring Error Procedural Error
17
Testing Irregularities (cont.)Procedural irregularities (examples)
Eligibility error Eligible students were not rated in one
or more domains Eligible students were not
administered the reading test (2-12) Ineligible students were assessed
IEP Implementation Issue A SPED student was provided or not
provided an accommodation Improper Account for Secure
Materials A rater, test administrator or CTC
misplaced a writing collection
Monitoring Error Test Administrator left a room
unattended when online tests were open
Students were not prevented from using cell phones or electronic devices
Procedural Error Test Administrator who was not properly
trained was allowed to rate students A TELPAS writing collection was not
submitted according to the assembly criteria
A test administration failed to provide a student with the correct student authorization for online testing
18
Reporting Test Irregularities Each person participating in the Texas student assessment program is responsible for reporting any violation or suspected violation.
Campus staff should notify their CTC or DTC if they witness an irregularity or suspect one has occurred
The DTC must contact TEA to report any incidents as soon as the DTC is made aware of the situation
Contact the DTC if unsure about whether an irregularity has occurred
19
General Information About Holistically Rated TELPAS Assessments
20
Each teacher selected to rate an ELL must
Have the student in class at the time of the spring assessment window
Be knowledgeable about the student’s ability to use English in instructional and informal settings
Hold valid education credentials such as a teacher certificate or permit
Be appropriately trained, as required by TEA
Rater Credentials
Raters may include: Bilingual teachers,
ESL teachers, General education
teachers, Special education teachers,
Gifted and talented teachers and
Teachers of enrichment subjects
TELPAS Rater Responsibilities A student’s TELPAS rater is the teacher designated
by the campus as the official rater of the student’s English language proficiency.
The student’s rater must rate the student in all domains for which the student is eligible. A student is not permitted to have one rater for some domains and another rater for other domains.
21
22
New & Returning Rater Definitions
• TEA requires that raters who have not rated during the last three years complete the BASIC
training.
23
Returning Raters
Online Basic Training Course
Online Calibration (Sets 1 and 2*)
If not calibrated: Supplemental Holistic Rating Training
Final Online Calibration (Set 3)
Spring TELPAS Administration Procedures TrainingAs a key part of this training, information from the TELPAS Rater Manual is reviewed to prepare
raters to proceed with online holistic rating training
Fall/Spring ELPS-TELPAS Foundation Training For teachers who will be trained as new TELPAS raters in the spring if they lack this foundation
Online Calibration(Sets 1 and 2*)
New Raters
*Set 2 required only if not successful on
Set 1
Holistic Rating Process
TELPAS Administration Procedures Training
As part of annual spring TELPAS administration procedures training, holistic rating training requirements are reviewed with raters, as well as information about how to access the online training and calibration components.
In addition, raters receive training on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc.
24
25
Online Training ComponentsOnline BASIC TRAINING Course – required for new raters and raters who have not completed rater training within the last three school years. Gives raters practice rating students in each language domains. Separated in two courses: (approximately 4-5 hours)
Grades Kinder-1st course – covers the four language domains
Grades 2nd – 12th course – covers listening, speaking, and writing Raters who have completed K–1
training but not 2–12 training will be new raters if they switch to 2–12 training, and vice versa.
26
Online Training Components (cont.)Online CALIBRATION Course – required for ALL raters. Each calibration set contains 10 students to rate and all holistically assessed language domains are represented in each set. (approximately 1 hour )
Grades Kinder-1st course – rate listening, speaking, reading and writing
Grades 2nd – 12th course – rate in listening, speaking and writing
Training & Calibration Grade ClustersRaters must know their assigned grade cluster to select the appropriate online training.
Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster.
27
A rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration.
28
Grade Cluster Organization for Online Training
Basic TrainingKinder - 1
Basic Training2 – 12
Calibration ClustersKinder - 1
Calibration ClustersGrade 2
Grades 3 – 5 Grades 6 – 8 Grades 9 – 12
29
Online Calibration ActivitiesRaters complete only as many sets as it takes to calibrate
Many raters will be able to calibrate on the first set of activities and many others will be able to calibrate by the end of the second set.
With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set
IMPORTANT NOTEIndividuals are not authorized by TEA to serve as raters unless they
complete the state-required training and calibration activities
30
Raters with Students in Multiple Grade Clusters that Include Grades 2-12
Raters are required to train and calibrate in the cluster in which they have the most ELLs.
As a best practice, these raters should also review the online basic training course practice activities in the additional cluster(s).
Example: A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.
Raters with Students in Multiple Grade Clusters that Include Grades K-1
Because of differences in the instructional content and rating rubrics, these raters must complete training and calibration for K–1 and at least one other cluster in 2–12.
Raters with more than one additional cluster should train in the cluster in which they have the most ELLs.
As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s).
31
Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.
New raters must complete the online basic training course BEFORE beginning calibration.
All raters have the option to review the online basic training course (which includes practice rating activities) before beginning calibration.
32
Preparing for Calibration Sets
Yes, raters should use their rating rubrics (PLDs) and refer to, as needed, information from the: online basic training course holistic rating PowerPoints produced by TEA TELPAS Rater Manual
Educator Guide to TELPAS33
Will raters be able to refer to any resources during calibration activities?
When and where do raters take the online training?Depending on campus arrangements, raters may either complete training and calibration during school hours, after school, or on weekends.
The online training system allows raters to access the training from any computer that meets the minimum system requirements.
Campuses inform raters of the details during annual administration procedures training.
34
Other Things to Know About CalibrationRaters affirm online that they will keep the contents of the calibration sets secure and confidential.
Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students.
Raters can work at their own pace, go back and review students, and change ratings as they work.
35
Other Things to Know About Calibration (cont.)Raters can exit and return later to finish. They click a “submit” button when they are finished with a set.
After completing a calibration set, raters immediately see results. Results show both the rating assigned by the rater as well as the correct rating.
Raters see annotations explaining the correct ratings. Raters should use the annotations to go back and review any incorrectly rated students.
36
Course and Calibration CertificatesRaters who take the basic training course get a certificate from the online Texas TrainingCenter after completing the course components.
Raters will receive a certificate of successful calibration when they calibrate.
37
38
Available Reports for Online Training and Calibration• TELPAS Returning Rater Planning Roster
– Uses records starting from the 2008–2009 school years to generate a list of previously trained raters (Basic Training)
• TELPAS Confidential Course Completion Roster– Lists online basic training course completion and in-progress status and performance by user for
the current year
• TELPAS At-a-Glance Training & Calibration Report– Provides a comprehensive list showing basic training course and calibration completions for all
registered users for the current year (includes time spent in each calibration set)
• TELPAS Confidential Calibration Summary Report– Provides calibration summary information by grade cluster (K–1, 2, 3–5, 6–8, 9–12) and for grade
clusters combinedhttps://texas.pearson.desire2learn.com/
39
Important Notes about Reports
• Personnel with administrative authorization are able to access reports to monitor TELPAS online training and calibration.
• The reports are updated nightly. • In order for users to appear in the correct reports, their location
(region, district, campus) must be up to date in the My Info section of the Texas Training Center.
• Users who have registered in the Training Center and have started but not completed a calibration set are not included in the summary reports. Only completed sets are presented in the reports.
• However, users who have started a training course will appear in the course roster as being “in progress.”
40
New Tools for CTC’s Coordinators and assistants with administrative access can modify the last name of a user send a user password reset email modify the email address of a user retrieve an email address associated with a user’s TrainingCenter account
retrieve a username associated with a user’s TrainingCenter account
see account status (active/deactivated) for all users.
41 TELPAS Additional RolesSupplemental Support ProvidersWriting Collection Verifiers
Supplemental Holistic Rating Training Raters not successful after sets 1 and 2 must receive supplemental
training The rater will meet with a campus-appointed supplemental support
provider (SSP)
CTC’s will notify the DTC who the SSP is by the designated district deadline
After the rater has received supplemental training, raters will be given an access code to proceed with the third and final calibration set
42
43
Criteria for Supplemental Support ProvidersA Supplemental Support Provider must have:
been a 2012-2013 fully trained rater in the applicable K-1 or 2-12 span
completed the recorded Web-based training, and calibrated successfully on either set 1 or set 2 in 2014
44
Training of Supplemental Support ProvidersTraining is delivered through a Web-based recorded presentation that is accessible from the coordinator resources section of the Training Center
Separate training for K–1, 2–5, and 6–12 Training will be available beginning January 27 Both new and returning supplemental support providers need to complete this
training
Campuses are encouraged to train at least one supplemental support provider for K–1, 2–5, and 6–12 to cover the needs of the school
Reviewing the number of raters unable to calibrate by end of calibration set 2 last year may help anticipate this year’s needs
45
Assemble and Verify Grades K-12 Student Writing CollectionsAn online training course on writing collection assembly and verification is available on the Texas TrainingCenter website
Each writing collection must contain at least five writing samples and must include one narrative writing about a past event and at least two academic writing samples from mathematics, science or social studies
Writing assigned on or after February 17, 2014 may be considered for the writing collection (writing assigned before February 17 is not eligible for inclusion)
Writing samples may continue to be gathered until March 26th, the date raters are required to turn in students’ rating to the campus coordinator
All writing should reflect a student’s current proficiency level
46
Assemble and Verify Grades K-12 Student Writing CollectionsAssembling Grades Writing Collections• Type 1: Basic descriptive writing on a personal/familiar topic• Type 2: Writing about a familiar process• Type 3: Narrative writing about a past event• Type 4: Personal narratives and reflective pieces• Type 5: Expository and other extended writing on a topic from
language arts
What Not to Include in a Collection (p. 23)
Verifying the Writing Collection Components (p. 24)
47
Responsibilities of Testing Coordinators for Holistic Rating Training• Ensuring raters understand the importance of being
properly trained on the holistic rating process
• Ensuring new and returning raters understand which training to take and for which grade clusters
• Monitoring that all raters complete their training requirements and receive supplemental training support if needed
48 Prepare to Rate
Holistic Rating Examples
49
Listening (pg. 28)The ability to understand spoken language, comprehend and extract information, and follow social and instructional discourse through which information is provided
EXAMPLES: Reacting to oral presentation Responding to text read aloud Following directions Cooperative group work
50
Speaking (pg. 29)The ability to use spoken language appropriately and effectively in learning activities and social interactions
EXAMPLES: Informal interactions with peers Oral presentations One-on-one interviews Classroom discussions
51
Reading K-1 (pg. 29)The ability to comprehend and interpret written text at the grade appropriate level
EXAMPLES: Paired reading Reading response journals Sing-alongs, read-alongs Independent reading
52
Writing K-1 (pg. 30)The ability to produce written text with content and format to fulfill grade appropriate classroom assignments
EXAMPLES: Journal writing for personal reflections Making lists for specific purposes Labeling pictures, objects, and items from projects Language experience dictation
53
Writing 2-12 (pg. 30)The ability to produce written text with content and format to fulfill grade appropriate classroom assignments
EXAMPLES: Descriptive writing on a familiar topic Narrative writing about a past event Procedural writing from science, math and social studies Expository and other extended writing from language arts classes
54
TELPAS Student Rating Roster
55
Browser-based TELPAS Student TutorialsAs in the past, tutorials are separate from testsRecommended for all students taking TELPASSeveral different test item formatsSince no sample items with test, practice with item formats and online interface is useful
Tutorials available at http://www.TexasAssessment.com/TELPAS-tutorials
56
Calendar of Events
57
Next Training!
TELPAS Training – Online Testing
Tuesday, February 25, 20148:30 – 11:30
CAB Cafeteria
*Bring final YEARS IN US SCHOOLS rosters