Innovations in Career-Tech Assessment
Presented by:Sandra Pritz, Senior Consultant
National Occupational Competency Testing Institute
Why should career-technical programs assess academic
skills?• State/national academic standards
demand attention.• High school students must pass
academic tests for graduation.• Workplace demands for technical
literacy are increasing.• Academics in context tend to be
student-friendly.
How Should Career-Technical Programs Assess Academic Skills?
• In the context of the students’ career-technical program
• In alignment with state and national academic standards
• In ways that help students translate to other tests they will need to take
• Embedded In their career-technical tests, such as end-of-course or program
Alternative assessments
• Ask students to perform, create, produce something
• Elicit applications of higher-level thinking and problem-solving skills
• Connect with tasks that are meaningful in terms of the related instruction
• Engage students in authentic real-world tasks
Adapted from A Practical Guide to Alternative Assessment (Herman,Aschbacher, and Winters, ASCD,1992)
Portfolio Assessment
• Involves assessing a collection of student work
• Establishes a purpose, criteria (for what gets put in, when, and by whom), and how the collection or individual pieces will be evaluated
• Engages students in assessment through gathering evidence of their progress and achievements that meet the criteria
Value Added to Career-Technical Programs by Tests
• External and effective• Critical competencies
benchmarked to industry standards
• Correlated to state and national academic standards
Effective Tests-
• are based on standards• provide transparent links between
standards, learning objectives, and learning activities
• help translate what is required into terms the learner can address
• complement projects, portfolios, alternative assessments
Critical Competencies Are Central
• Students should recognize them as learning objectives with standards, move toward self-assessment.
• Teachers should teach to them in learning activities.
• They should be assessed—in multiple ways—with the results used to help focus instruction, for the group and for individuals.
Process to Use Multiple Tests to Improve Instruction
• Written tests—practice at all levels by including in assignments; mirror types of items on HSTW tests, include academics embedded in the technical industry/program.
Electronic Technology WrittenStudents must be able to comprehend the following:
•Read diagrams
•Recognize symbols
•Apply Ohm’s LawMath Calculations needed:
•Solve parallel and series circuits.
•Solve algebra equations
•Manage inverse relationships
Performance Tests
- Practice by including scenarios in assignments
- Mirror real job tasks, time, tool selection, procedures
- Open-ended, can apply a rubric, and can easily assess at higher levels
- Some aspects yes/no, some scaled, some qualitative judgment by expert
Precision Machining Performance
1.You will receive a piece of cold rolled steel
1.125” diameter x 6.62 long.
2.Following industrial safety rules, machine the part on the lathe according to the specifications given on the drawing provided. Check tools for condition and sharpness.
Requires student to: -evaluate drawing to develop a work plan -select speeds and feeds - read blueprints
Coupling Written and Performance Tests
• Reflects reality better than one or the other alone
• Provides for multiple forms of assessment• Gives all students a chance to succeed • Enables mutual reinforcement of learning• States and schools using NOCTI tests
have documented improvement in instruction from using analyses of test results.
Career and Technical Skills
AcademicSkills
Academic Skills
+ Career Technical Skills
Comprehensive Assessment
Academic Skills Measured in Context
Assessing Academic Skills in Context
• NOCTI tests of occupational competencies also test some embedded academics.
• Math, science, and language arts are required to complete most occupational items and performance tasks.
Needs in Reading
• Comprehend and interpret the materials of the occupational field
• Compare, analyze, synthesize technical materials into a new set of information
Needs in Math and Science
• Use analysis techniques• Solve problems that require
integration of more than one math concept and/or multiple steps
• Estimate and check answers for reasonableness
• Solve open-ended problems
Science -- Carpentry
• 4% of the written test items and 14% of the jobs of the performance test match NSES science standards at the 9-12 grade level.
• Example question: Concrete wall forms must be made __________
(A. strongest at the top, B. strongest in the middle, C. strongest at the bottom, D. uniform throughout).
NOCTI Academics Correlation Project
• 7 NOCTI job-ready tests correlated to academic standards for math, science, and language arts- National standards (NCTM, NSES,
NCTE)- State standards (NY, CA, FL)
• Several business/industry tests correlated to national standards
Academics Identified
• Written tests contain embedded academics that match standards in all three disciplines.
• Performance tests contain a higher percentage of embedded academics than written tests, and the items tend to be at higher levels of proficiency.
Employability Skills
• Competency lists like SCANS can be used to embed matching employability skills in tests
• Bloom’s taxonomy* can be used to embed higher order skills in tests
* Bloom’s taxonomy progresses from knowledge to comprehension,application, analysis, synthesis, and evaluation
Bloom’s Taxonomy -- Carpentry
Written Performance
Knowledge 37% 14%
Comprehension 25% 0%
Application 27% 14%
Analysis 10% 0%
Synthesis 1% 72%
Evaluation 0% 0%
SCANS Skills -- Carpentry
Written46%
Performance86%(1 or
more)Resources 6%
Information 8%
Systems 5% 50%
Technology 27% 86%
Sample of Carpentry Written Test Item with Technical and Academic Content
at Proficient Level
If a benchmark is established at 100’-0” and the bottom of the foundation is at 96’-4”,the bottom of the foundation is _______ the benchmark. (A. 6’-4” above, B. 3’-8” above, C. 6’-4”below, D. 3’-8” below)
Math: proficient, 2 national standard matchesReading: basic, 1 national standard matchBloom’s: analysis SCANS: acquires and
evaluates information
Sample Carpentry Performance Job At Advanced Technical Level
with Integrated Academics
Using the appropriate tools, equipment, and material, install an interior window frame in the rough opening provided. Case the top and part of both sides of the frame using mitered corners. Use a table saw or power miter box to cut trim.
Math: proficient; 4 national standard matchesReading: advanced; 1 national standard matchBloom’s: synthesis SCANS: selects technology
Ways NOCTI can help
• By providing performance as well as written tests, NOCTI can help assess academics at higher levels.
• NOCTI can respond to the need to reinforce academic and employability instruction through assessment, can customize tests and test reports to isolate academics scores.
• NOCTI can provide states the service of cross-correlating its tests to their state standards.
Ways NOCTI can help (cont.):Workplace Readiness Test
• NOCTI Workplace Readiness Test measuring employability skills is being up-scaled
• A large and stellar Subject Matter Expert Team has rated the critical competencies
• Career Cluster Foundation Skills will be assessed
Ways NOCTI can help (cont.):Workplace Readiness Test
• Employability skills are focus of NOCTI best-selling Workplace Readiness Test since 1996.
• Up-scaled test in process to meet changed needs of business & industry– Research and synthesis of standards literature (e.g.
MSSC, SCANS, EFF)– Review by >30 leaders, 3 focus groups– New assessment to be developed by fall– Will be formatted to align with career clusters
For Tech Prep Programs
• NOCTI’s critical competency lists can be used to document curriculum alignment for secondary and post-secondary articulation.
• NOCTI can validate student mastery of competencies at the secondary level for transition to and possible credit at the post-secondary level.
States Career Cluster Initiative Moves to Assessment Stage
• NASDCTEc has named NOCTI a developer and the provider of its cluster, pathways, foundation tests.
• NOCTI is administering the test developed for the National Health Sciences cluster.
• NOCTI is developing 3 pathways tests in construction for PA, to be used nationally.
• NOCTI is developing a Workplace Readiness foundation assessment.
What Do I Receive from NOCTI?
• Score Reports– Coordinator’s Report– Individual Student Reports– Customized Reporting Available—
e.g. integrated academics
• Statistical Information– Group Data– Site Data– State Data– National Data
• Student Certificates
• Personalized Contact Assistance
All results are broken down into
duty, category and total scores.
Note:
Job Ready Welding Written Assessment
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Ther
mal
Cut
ting
Proc
esse
s
Hea
tTr
eat/M
etal
lurg
y/D
isto
rtion
Con
trol
Wel
ding
Ter
ms/
Blue
prin
tR
eadi
ng
Wel
ding
Insp
ectio
n
Wel
ding
Con
sum
able
s
Safe
ty
Pow
er S
ourc
e Pr
inci
ples
Oxy
fuel
Wel
ding
&Br
azin
g
Shie
lded
Met
al A
rcW
eldi
ng
Gas
Met
al A
rc &
Flu
xC
ored
Arc
Wel
ding
Gas
Tun
gste
n Ar
cW
eldi
ng
Mea
surin
g In
stru
men
ts
Wel
ding
Qua
lifica
tions
&Pr
oced
ures
Wel
ding
Cos
ts
Tota
l
Pre-Test Scores Post-Test Scores
Pre and Post Test Gain Analysis
Contact NOCTI for more information and
to discuss your interests---
www.nocti.org or 1-800-334-6283
Thank You