PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236 Contacts: Joe Tornabene Phone: (908) 451-0355 Email: [email protected]Mark Marchuk Phone: (908) 751-5224 Email: [email protected]1 PACE FastScore® Overview PACE’s FastScore® is a Windows based computer system that tightly integrates with the Cardiff TeleForm Automated Data Collection Suite to provide a dynamic turnkey solution regarding the design, scanning, character recognition, auto-exporting, scoring and reporting of test results. Any TeleForm designed test , exam or questionnaire can be introduced to FastScore® via a TeleForm auto-export for immediate scoring and reviewing of reports with no custom scripting required . FastScore® also provides a variety of exports allowing for the automated updating of student grade reporting systems, like SIS, SIRS and SASY and a variety of other databases, data warehouses and information management systems. FastScore® thereby is a program that enables you to interact with exam results fast – identifying what is taught well or poorly, and who is taught well or poorly. FastScore® consists of three core Application Units: MS-Access/SQL Database • Holds answer keys • Holds results of examinations taken TeleForm Auto-Export Scripts: • “PaceRege_VBA” is the auto-export available in the TeleForm drop-down choices (found within the TeleForm Designer). • After exams are Optically Scanned, Evaluated and Verified in TeleForm, the export script saves the data from the TeleForm into the MS/SQL Database. • An optional TeleForm script (also created/distributed by PACE ) allows for the intentional skipping of questions-parts when in the TeleForm Verifier (therefore greatly reducing redundant effort of tabbing through blank fields while operating the TeleForm Verifier). FastScore® Windows Application: • Introduces new exams/answer-keys into the MS-Access/SQL DataBase • Immediately Scores scanned-in exams which were verified and auto-exported from TeleForm (no custom export scripting required ). • Provides for the Viewing and Printing of Analysis-Reports. • Provides exports for Student Scores to back-end database systems (E.g.: the SIS grade reporting system, BOCES Data Warehouse, etc.). • Exports the Scores as textual files. (Fixed formats, CSV formats, Data-Grid-SaveAs) • Allows viewing of the original exam’s image (.TIF) while in the FastScore graphic user interface when reviewing the scoring results. • Allows for Exports and Imports of Answer Keys. • Imports .CSV tables into the LINEAR and MATRIX scales of an Answer Key • Imports field-value references that decode IDs into full names (E.g. IMPACT Regents format for NY State Regents Examinations).
Transcript
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
PACE’s FastScore® is a Windows based computer system that tightly integrates with the Cardiff TeleForm Automated Data Collection Suite to provide a dynamic turnkey solution regarding the design, scanning, character recognition, auto-exporting, scoring and reporting of test results. Any TeleForm designed test , exam or questionnaire can be introduced to FastScore® via a TeleForm auto-export for immediate scoring and reviewing of reports with no custom scripting required. FastScore® also provides a variety of exports allowing for the automated updating of student grade reporting systems, like SIS, SIRS and SASY and a variety of other databases, data warehouses and information management systems. FastScore® thereby is a program that enables you to interact with exam results fast – identifying what is taught well or poorly, and who is taught well or poorly. FastScore® consists of three core Application Units:
MS-Access/SQL Database • Holds answer keys • Holds results of examinations taken
TeleForm Auto-Export Scripts: • “PaceRege_VBA” is the auto-export available in the TeleForm drop-down
choices (found within the TeleForm Designer). • After exams are Optically Scanned, Evaluated and Verified in TeleForm, the
export script saves the data from the TeleForm into the MS/SQL Database. • An optional TeleForm script (also created/distributed by PACE ) allows for the
intentional skipping of questions-parts when in the TeleForm Verifier (therefore greatly reducing redundant effort of tabbing through blank fields while operating the TeleForm Verifier).
FastScore® Windows Application: • Introduces new exams/answer-keys into the MS-Access/SQL DataBase • Immediately Scores scanned-in exams which were verified and auto-exported
from TeleForm (no custom export scripting required). • Provides for the Viewing and Printing of Analysis-Reports. • Provides exports for Student Scores to back-end database systems (E.g.: the SIS
grade reporting system, BOCES Data Warehouse, etc.). • Exports the Scores as textual files.
(Fixed formats, CSV formats, Data-Grid-SaveAs) • Allows viewing of the original exam’s image (.TIF) while in the FastScore
graphic user interface when reviewing the scoring results. • Allows for Exports and Imports of Answer Keys. • Imports .CSV tables into the LINEAR and MATRIX scales of an Answer Key • Imports field-value references that decode IDs into full names (E.g. IMPACT
Regents format for NY State Regents Examinations).
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Available Analysis’ / Reports: (see below screen shots) • Primary Roster Grid-List of Scores by Student or by Teacher -
showing Scaled and Raw points by Part(s), with net Average and Standard Deviations per column w/variety of sorts and filters.
Pop-Up Per Student - on how that student did per question (right/wrong; scaled, raw)
Pop-Up showing .tif image of Student’s Test Answer Sheet • Average Per Question and Standard Deviation - showing how well the
students did per question. E.g.: which questions were hard, which were easy, what the standard deviation was per question, w/percentage hits per question.
• Average Per Question by Gender – shows possible gender biases. • Range Analysis of Scaled Scores – allowing analysis/views on 5 core
ranges of scores: 0-54, 55-59, 60-64, 65-84, 85-100 With a Detail Student Roster per Range
• Analysis by Teacher, by Student within specific scoring ranges.
Updates: • Ability to fix a student’s answer on the Roster-Grid-List of Scores.
Answer Key: • Ability to relate groups of questions as logical exam parts and/or
categories. • Option to have different earned points per a question's answer;
thus allowing a weighted-rate, E.g.: option to have more than one answer get you earned points, wherein answer (A) might get you 3 points, (B) might get you 2 points, (C) might get you zero pts.
• Option to turn on/off a part's questions in the FastScore® Answer Key; thus (while in the TeleForm Verifier) not having to redundantly tab through parts and questions that were not intended to be scored at that particular point in time.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Scoring techniques: (see below screen shots) • Multiplier Constant: Scaled Score = Raw Pts x Constant • Linear: Scaled Score = Raw Pts referenced against a table
E.g.: Raw of 7 might relate to a 75; Raw of 8 might relate to a 81; Raw of 10 might relate to a 100.
• Matrix: Scaled Score = Raw-group-of-part(s) intersection with Raw-group-of-another-part(s) E.g.: If you earned 16 points in Part 1 (Math), and 23 points in Part 2 (English), the matrix intersection at "16 by 23" may give you an earned points of 73.
Exports: (see below screen shots) • CSV - Export-Standard (Note: CSV stands for Comma Separated Value) • CSV - Export-All-Scores • CSV - Export-All-Scores/Responses/Keys • Fixed - Export - Grading Spreadsheet (Fixed length LRECL 82) • SIRS – Student Information Repository System • Save-As - On the Whole Student Data-Grid
Demographics: (see below screen shots) • Decoding Student-ID's: name, gender, ethnicity, district, school, etc. • Import an external textual REGENT-format into FastScore®. • Day of Test (walk-in) entry per student (via a FastScore® dialog pop-up).
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
The following pages contain a variety of Screen Shots intended to provide a visual overview of some of the functional attributes available in the PACE FastScore®
program and its integration with Verity’s TeleForm software.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Designing Regents Answer Sheet Forms (with TeleForm Designer)
Example: Below is an example TeleForm showing both CHOICE and OPEN ENDED questions. Each exam can have its own unique design, the only requirement being to name the fields using the agreed upon names, especially: “district_id” and “student_id”.
Example “Choice” TeleForm Example “Open Ended (OE)” TeleForm
PACE FastScore®
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Entering Answer Keys into the DataBase using a “MULTIPLIER” Scale Type
Use a Scale Type of: MULTIPLIER when each question’s raw-points are multiplied by a constant (E.g.: if the multiplier is 2, then the total scaled score is the sum of the correct-raw-points multiplied by 2). Save an answer key before having TeleForm Reader evaluate any images. Also before evaluations, make sure the Max-Questions-to-Score is correct for each Exam-Part. Note: If a question is Open Ended, enter “OE” as the correct answer, and enter the maximum points allowed in the Points column.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Entering Answer Keys into the DataBase using a “LINEAR” Scale Type
Use a Scale Type of: LINEAR when you relate the total tallied raw score to a two column table, where the 1st column is the raw score, and the 2nd column is the examination’s scaled score.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Entering Answer Keys into the DataBase using a “MATRIX” Scale Type
Use a Scale Type of: MATRIX when an exam has two parts, (like CHOICE and OPEN-ENDED-essay), and relate each part’s raw score to a matrix table, where the intersection of the two parts raw-scores is the examination’s scaled score. For each question, you enter what exam-part the question tallies into. Note: If a question is Open Ended, enter “OE” as the correct answer, and enter the maximum points allowed in the Points column. (See Exam’s Answer Sheet box)
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
TeleForm Verifier (Operator Reviewing the Character Recognition)
Below is an example of one TeleForm page in a batch of pages in the TeleForm Verifier correction mode. A user/operator will tab or tap the enter key to move to the next questionable character allowing them to make the final decision. After all fields have passed review, and all pages of a batch are completed, the batch will be in status: “Batch ready to be committed”. (See next page)
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
TeleForm Verifier (Operator in Batch Management Screen)
Below is an example of a batch that has passed review, (E.g.: all the pages of the batch that needed review have been reviewed by an operator), and the batch is now in “Batch ready to be committed” status. It is either manually committed: (batch options > commit), or the TeleForm system is set to auto-commit all “Batch-ready-to-be-committed” status-ed batches. Once a batch is committed, the examination data and image references are ready to be processed by the “PACE FastScore” application. (See next page)
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
The below screen, (from PACE FastScore > menu > application > score tests), allows the user to select which examination to score. This is typically a recent exam that has been auto-batch-committed via TeleForm.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
The next step after having selected which exam-test to score is to: Click on Score Answer Sheet, and all data related to this exam will immediately be scored. The student tests are scored according to its answer key, and the data is now displayed in the FastScore graphic user interface (see below) and ready for printing and exporting. Note: if you have previously imported “IMPACT” demographic data regarding student, teacher, school, district, course or section ID names, then those student’s demographic values will be displayed. Also available at this time are the variety of Exports (see below Exports Group).
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Report: Average and Standard Deviations per Question
You can view this Average and Standard Deviation Analysis Report by clicking the “Avg/Qu” button on the Score-Dialog. This quickly shows which questions were easy, and which were hard, and the answer hit-percents per question’s picked-answer.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
You can view this Average Scaled Grade Percents Report by clicking the “Ranges” button on the Score-Dialog. This quickly shows how well the students did per percent-range. From here, you can get a detail list of those students in each range (see below Detail Student Roster Report buttons).
PACE FastScore®
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Report: Student Roster Detail List in the (65-84) Range
This report details the students that had a scaled range between 65-84. Note the subsequent sorts and filters that can be used here to continue to view these students in a variety of ways.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
The regional version of the “Average Per Question” allows you to compare your district’s averages against a set of districts-(region) so you can quickly see how your district compares to others. (Note: The Range-Report also has a regional option).
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Decoding: IMPACT Format Technique Using PACE FastScore
By loading a fixed length IMPACT Format reference file, you can auto-populate certain full-names according to the district_id+form_ID and student_id match. (See the menu item: "Import IMPACT Decodes"). Note: The IMPACT decode demographic file should be in the standard Regents format, which is a fixed position 238-length file. Field Name Width Position =============== ===== ================= BEDS District 6 1-6 Required BEDS Type 2 7-8 Required BEDS Building 4 9-12 Required School Year 4 13-16 Required BEDS Title I 1 17 Student Funding 4 18-21 Student ID 9 22-30 Required Student Last Name 22 31-52 Required Student First Name 15 53-67 Required Student Middle Name 15 68-82 Student DOB yyyymmdd 8 83-90 Required Student Grade Level 2 91-92 Required Student Gender 1 93 Required Student Ethnicity 1 94 Student Poverty 1 95 Student Population 1 96 Student Mobility 1 97 Student Disability 1 98 Student Pg m Svc 1 99 Student Lvi Integrat 1 100 Student Beds of Svc 6 101-106 Student LEP 1 107 Student LEP Years 1 108 Assmt Subject Area 1 1 109 Assmt Measure 1 3 110-112 Assmt Lauguage 1 2 113-114 Assmt Amin Yr 1 4 115-118 Assmt Amin Mo 1 2 119-120 Assmt Score 1 3 121-123 Assmt Std Met 1 2 124-125 Student:Course-Section 20 126-145 Required eg: 0999-999 Teacher Name 25 146-170 Required Assmt Filler 68 171-238
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Log-In Group Security Using PACE FastScore Security: Combines Userid-Password-Groups (E.g.: valid Districts) Who can interact with a Teleform-ID's AnswerKey and its committed exam tests is related to which user has set up the AnswerKey. (E.g.: A Teleform-ID's answer key gets the Group-Code of the user who sets it up). Each Userid is assigned to a Group Code and associated with a list of valid districts; ergo users can only interact with those Teleform-ID's Answer-Keys, and any of their committed tests that have their Group-Code (I.e.: A UserId will only be able to interact with exams that are related to their valid districts. Note: To add or change Userid-Groups, you must logon with an “AllGroups” Group Code, usually called: ADMINISTRATOR. This group-code can interact with all Teleform-ID Answer-Keys.
PACE Business Solutions 297 Route 72 West, Suite35-PMB238, Manahawkin, NJ 08050-2890 Phone: (908) 451-0355 Fax: (908) 751-5236
Maintaining a list of available Districts Using PACE FastScore District ID Maintenance: The RIC administrator uses this dialog to maintain the available list of districts.
