STEP 4 – EVALUATION

“Are we doing what we said we would do?”

Objectives

A. PROCESS OF EVALUATION 1. What does Evaluate Mean?2. Benefits of Evaluation3. “Documentation is Key” – Many means to

document evaluation4. Choose method to best match elements 5. Program Requirements for Evaluation (Step 4)6. Negative Evaluation Results7. Evaluation – “Best Practices”

B. DISCUSS EVALUATION METHODS - 10 METHODS

5 Steps to Managing Health & Safety

1. Written standard

2. Communication

3. Training4. Evaluate or Evaluation plan

5. Acknowledge success and make improvements

Achieving an ElementContinuous Improvement

Evaluate…What does this mean?

1. Verifying that the company’s Standards are still current and up-to-date to both:– Legislation AND – Company “Rules”

2. Verifying that what is actually being done in the

workplace meets the standards and company rules established in Setting Standards (Step 1).

“Is the workplace doing what it said that it would do?”

Benefits of Evaluation

• Assesses implementation of your element• Assesses what’s working & what’s not working –

“gaps” in your health & safety program• Addresses workplace culture issues i.e. employee

participation, supervisor support, functional JHSC• Opportunity to review records re: standards,

communication & training• Reviews your accident/incident performance• Allows for recommendations to improve H&S

Program

Evidence of Evaluation

• Need to develop a Method of Recording/Documenting Evaluation

• Not restricted to examples provided, you may have other effective ways to record/document

Benefits of different methods

• Consider the element being evaluated• Observations may be effective when

evaluating skill-based elements:– i.e. SOPs, PPE, Fork-Lift, Machine Guarding,

Housekeeping, etc.

• Quiz/Survey/Interview may be effective when evaluating knowledge-based elements:– i.e. worker comprehension of H&S Policy, Roles &

Responsibilities, ESRTW, Hazard Reporting, JHSC/Reps etc.

Program Requirementsfor Evaluation (Step 4)

• Document all forms of evaluation – incorporate into existing methods

• Aim to complete Evaluation by year- end. – If not, must complete by March of following year

(prior to spot checks starting).

• But MUST have written plan of “how” element will be evaluated and completed by March to put “Y” on Year-end Report

Negative Results

• The results of the Evaluation indicates “Negative Results”– For this element, “the company is not doing what the

Standard expects it to do”

• Is this still Evaluation (Step 4)?

YES!

• Negative results (or less than positive results) provide for Opportunities for Improvement.

Best Practices

• Write method of evaluation in Standard ( who, what, how, where & when)– Include benchmarks in your written standards

• Ask supervisors to keep log books/checklists if involved in worker observations

• Ask JHSC to review elements; Record in JHSC Minutes & Post

• Review elements at Management Meetings; Address recommendations; action plan to improve; & Minute meetings

• Use existing methods to DOCUMENT evaluation.

Step 4 - Evaluation

10 Best Practices/Examples/Samples1. Interview/survey employees2. Quiz 3. Scheduled workplace/site/shop inspections or

tours4. Joint Health & Safety Committee5. Management Team Review6. Internal audit7. Observations8. Checklists9. Injury/incident statistical analysis10.Third party review/audit

Interview/Survey Employees

1. Determine your objectives 2. Address all elements simultaneously3. Identify required amount of participation 4. Develop list of questions relating to elements

and your objectives5. Questions can either be asked verbally one-on-

one with staff or by handouts

SAMPLE Interview/Survey EmployeesDo you know where the JHSC Yes Where:_______________________________Don’t know

Who do you notify if you need first aid treatment?Yes Who____________________ Don’t know

You suffer work place injury and will be absent from Work. List Responsibilities? __________________________________________________________________________________________________________________________

Employee QuizDetermine your objectives

Address all elements simultaneously

Determine participation level 100% (mandatory or voluntary)

Develop questions specific to objectives and elements

Questions?? quiz format, multiple choice, true/ false, fill in the blanks, etc.

SAMPLE: Quiz ExamplesIs PPE checked regularly for wear or defects, and replaced immediately if necessary?

Yes NoDon’t know

The hazard reporting procedures allows for immediate reporting and corrective action without waiting for the next round of regular inspections.

Yes NoDon’t know

Is the Health & Safety Representative selected by the owner?Yes NoDon’t know

Scheduled Workplace/Site/Shop Inspections

or Tour

• include all 5 elements in monthly inspection report/checklist

• ensure JHSC/representative or supervisor understands

• Ask questions

Scheduled Workplace/Site/Shop Inspectionsor Tour

• See Appendix A SAMPLE Workplace Inspections Checklist(To be used in conjunction with inspection

report)

Joint Health & Safety Committee

• JHSC must be an active participant in the safety group initiatives since the start of year

• Discuss at meetings/record in minutes

• Provide JHSC all pertinent information: meeting minutes, inspection reports, injury statistics, supervisor’s comments, training records, etc

• Discussion to identify areas of improvement for next year and areas of success

Joint Health & Safety Committee

• See SAMPLE Appendix BSAMPLE JHSC OR MANAGEMENT MEETING

MINUTES

Management Review

• Management & supervisors must be aware and involved in safety group initiatives during the year

• Hold meetings ( quarterly updates) • Present to management team the progress of

the 5 elements and necessary documentation• Discussion

• Areas of improvement• Areas of success• Next years elements

Management Review

• See SAMPLE Appendix BSAMPLE JHSC OR MANAGEMENT MEETING

Internal Audit

• Measure organization against external standards– Workwell Core Audit or corporate audit, CSA, other

guidelines

• Use entire audit or use portions of audit that apply to your element selection

Internal Audit

Sample Guidelines from Workwell Core Audit:

• Early & Safe Return to Work– Roles & responsibilities– Contact with injured worker– Medical monitoring and

treatment– Provisions for modified work– Re-integration of the worker

to regular work– Reporting requirements to

WSIB

• Action Required with Target Dates– policy required:

including roles & responsibilities

Observations

• Decide who and what will be observed• Physically observe activity being done

– lock-out/tag-out, Fork truck, Emergency Evacuation, Confined Space

• Conducted by supervisor and/or JHSC• Provide immediate feedback to employee• Can be used in conjunction with annual

performance reviews

SAMPLE Observation ExampleForklift Procedures Date:___________Observers’ Name: _________________ Operator: __________________________

Pre-Operation CheckGeneral Operation: operating around personnel, reversing, parkingload handling: pick-up, placement, stacking, re-stockingloading /unloadingoperational maintenance: refueling

Immediate Feedback:

Operator Comments:

Recommendations:

Checklist Format

• Identify in advance how and what will be evaluated

• Excellent resource document for year-end documentation

Checklist

• See SAMPLE Appendix C- Evaluation Form

Safety Group Evaluation Form Initial Year or Maintenance ElementSafety Element :

____________________________________________________________________Instructions: 1. Reviewers are to familiarize themselves with the Standard set in the

Working Instructions and determine that it is current, relevant and meets legislative requirements.

2. Reviewers are to conduct an evaluation to see that Standard is known and practiced in the shop.

3. Reviewers will observe practices, or interview sample employees to see that details in the element are known, employees understand the training that they were given, and that the work is being done according to the standard for the element.

4. Evaluate using at least one of the following:

APPENDIX D

Sample Safety Group Element Evaluation Form

Safety Group Name       

Firm Name      

  Date (dd-mmm-yyyy) 15-Dec-2004 Evaluation Report

RTW Self-Assessment Guide

RTW Self-Assessment Guidehttp://www.wsib.on.ca/wsib/wsibsite.nsf/LookupFiles/DownloadableFileRTWSelf_AssessmentGuide/$File/RTWSelfAssessment.pdf

Injury/Incident Statistical Analysis

• Opportunity to measure the impact of completed elements on the injury/incident rates in the workplace

• Consider using:– a historical comparison against previous years– a comparison by locations or branches within

business– a comparison to your rate group

SAMPLE Injury/Incident Statistical Analysis

• Analyze key indicators in your workplace:– Lost-time injuries– No lost-time injuries– First Aid– Near misses– Property Damage– Duration of WSIB claims

0102030405060708090

1stQtr

3rdQtr

2003LTI's

2004LTI's

2005LTIS

Third Party

• Corporate auditor, health & safety association, safety group member, health & safety professional ( ergo, hygienist)

Benefits• Address all elements at same time• Auditor will provide insight and recommend

improvements

Questions?

Shelley WallShelley_wall@wsib.on.ca

416-344-5014