Management of SQA’s Quality Assurance

Standards

Matthew McCullagh

Quality Manager

Barbara Irvine

Quality Enhancement Manager

Welcome

欢迎

Tuesday 2nd September 2014

9.00 Welcome

9.15 Introduction

9.45 Management of successful assessments

10.30 Break

10.45 Management of successful assessments

12.00 Lunch

1.00 Management of Internal Quality Assurance |standards

2.45 Break

3.00 Management of Internal Quality Assurance |standards

4.30 Completion of Action Plan

5.00 Close

SQA’s Approach to QA

SQA launched it’s approach to Quality Assurance in China in

August 2012.

•Today is about helping you

•Consolidate your knowledge of SQA’s Approach to Quality

Assurance

•Identify areas of your Quality Systems which may require

review

•Support you in the delivery of SQA qualifications

•Help you to understand how to prepare for successful

systems verification

Support available

On line guidance is available for centres

•http://cn.sqa.org.uk/Quality Assurance

Quality Criteria

• Five Categories: • Management of a centre

• Resources

• Candidate Support

• Assessment & Verification

• Records/Data Management

• All signposted to four QA processes

• All pre-rated as High/Medium/Low impact

• All supported by possible sources of evidence

SQA’s Approach to Quality Assurance: The Four Processes

• Systems Approval

• Qualification Approval

• Systems Verification

• Qualification Verification

High/Medium/Low Impact Criteria

• All criteria are important

• However, some are more important than others

• Those that are most important are rated as High Impact

criteria

• Those that are less important are rated as Low Impact

criteria

• In between these two are medium impact criteria.

• Failure to meet a High Impact criteria will have a greater

impact on the outcome of the visit

Sufficiency of evidence

• No Systems Verifier or Qualification Verifier can change the

impact rating of a criteria.

• For each criterion you verify, you must decide whether the

centre has presented you with:

• Sufficient evidence to meet the criterion

• Some, but insufficient evidence to meet the criterion

• No evidence to meet the criterion

Good Practice and Guidance

• As part of the verification visit, you should identify to SQA

Quality Assurance staff any good practice you feel that is

evident in the centre.

• Systems Verifiers will also give advice and guidance to

help centre staff improve the delivery of our qualifications

• This is a mandatory part of the role of Systems Verifier

and the Qualification Verifier

Sufficiency of evidence

• We will support you and help you to understand what sort

of evidence centres can provide to meet each criterion

• Once we have made your decision regarding the

sufficiency of evidence, SQA staff will then calculate our

level of confidence

• This will depend on the impact level of the criteria and the

sufficiency of the evidence presented

Confidence Statements

In relation to Qualification

Verification

Confidence

Statement

In relation to Systems Verification

High level of confidence in the maintenance of

SQA standards within this Verification Group

High Level of Confidence High level of confidence in the systems that support the

maintenance of SQA standards within this centre

Broad confidence in the maintenance of SQA

standards within this Verification Group

Broad Confidence Broad confidence in the systems that support the

maintenance of SQA standards within this centre

Reasonable confidence in the maintenance of

SQA standards within this Verification Group,

although moderate risks exist within the

following categories: (insert appropriate category) (insert appropriate category)

Reasonable Confidence

Reasonable confidence in the systems that support the

maintenance of SQA standards within this centre, although

moderate risks exist within the following categories: (insert appropriate category) (insert appropriate category)

Minimal confidence in the maintenance of SQA

standards within this Verification Group as

significant risks exist within the following

categories: (insert appropriate category) (insert appropriate category)

Minimal Confidence

Minimal confidence in the systems that support the

maintenance of SQA standards within this centre as

significant risks exist within the following categories: (insert appropriate category) (insert appropriate category)

No confidence in the maintenance of SQA

standards within this Verification Group as

severe risks exist within the following categories: (insert appropriate category) (insert appropriate category)

No Confidence

No confidence in the systems that support the maintenance

of SQA standards within this centre as severe risks exist

within the following categories: (insert appropriate category) (insert appropriate category)

How Confidence Statements are calculated

• Confidence statements are given for the outcome of each

Systems Verification and Qualification Verification visit for

each of the categories of criteria verified

• These are based on the sufficiency of the evidence provided

to the you

• A summary statement of confidence is also given, based on

the confidence statements for each category

What if non-compliances are identified?

• Where there is No evidence or insufficient evidence to

meet a criterion, you must agree with SQA staff, during the

visit:

the action the centre needs to take in order to become

compliant.

The evidence they need to produce

Where to send this evidence

The date by which the action must be taken

What happens then?

• SQA will ask the Systems or Qualifications Verifier to

consider the evidence you, the centre sends us and ask

them to consider whether it is Sufficient or insufficient.

• Depending on the sufficiency of the evidence, the

Confidence Statement will be re-calculated.

Sanctions

• If the outcome of a verification visit results in a Confidence

Statement of Reasonable, Minimal or No Confidence, SQA

staff may decide to place a sanction on a centre until our

confidence increases to Broad or High.

• Our confidence may decrease if a centre fails to meet action

points agreed during a verification visit.

• Sanctions will only be used where necessary.

Sanctions In relation to Qualification

Verification

Confidence

Statement

In relation to Systems Verification

High Level of Confidence

Entry in Action Plan Broad Confidence

Entry in Action Plan

Suspension of specific qualification

certification: by verification group by qualification Suspension of approval application by

verification group Suspension of direct certification claim status: by verification group by qualification

Reasonable Confidence

Suspension of all existing qualification approval Suspension of centre certification Suspension of certification for all qualifications with

assessed components Suspension of qualification approval application - whole

centre

Suspension of specific qualification approval: by verification group by qualification

Minimal Confidence

Withdrawal of all existing specific qualification approval

Withdrawal of specific qualification approval: by verification group by qualification

No Confidence Withdrawal of centre approval

Benefits of this approach for centres

• Open and transparent to all

• Clear guidance available to verifiers

• Proportionate response where issues are identified

• Focused on Good Practice and improvement as well as

compliance

Questions

Approach to Quality Assurance

Welcome Back

欢迎回来

Quality Systems

•Why is it important to have good quality systems in place?

•What constitutes a Quality System?

Quality System

• Documented policies and procedures – Quality Manual

• Paper v electronic system

• Security of Quality Manual

• How do you effectively communicate changes in policies

and procedures to staff and/or students?

• How do you review/update your Quality System?

Internal verification

• Documented three stage process

• - Pre Delivery

• - During Assessment

• - Review

• Should underpin your centres programme of delivery

• Ensures standardisation of approach and consistency

across all SQA qualifications delivered in your centre

Pre Delivery

• Basically a check that you have everything in place to

delivery a particular qualification/unit

• Qualified staff in place who know the unit they are delivering

• Resources – reference and learning materials

• Assessment materials that have been checked by assessors

and internal verifiers

• Evidence this using pre-delivery checklist

During Assessment

• Internal verifiers are in place to support assessors

• Confirm assessor decisions are correct and there is a

consistent approach when more than one assessor per unit

• Arrange standardisation meetings

• Observe assessors and provide feedback on performance

During Assessment (2)

• Maintain records of assessment and internal verification

i.e. Master folder containing

- validated assessment materials

- checklists/marking schemes

- student records

- records of all sampling activity

- records of observations on assessment

- feedback to assessors

- minutes of standardisation meetings

During Assessment (3)

• Consider where to direct internal verification effectively –

risk/proportionality

• What factors might influence this decision?

Review

• What went well?

• What did not go so well?

• Were the materials in use fit for purpose?

• Is recording documentation completed and fit for

purpose?

• What do we need to change?

• Follow documented process to ensure relevant policies

and procedures are updated to reflect any changes made

Inductions

•Staff Induction

•Student Induction

•Both are very important and do not necessarily only take

place at beginning of course/session

Induction (2)

• Content very important

• Staff and Student handbooks are important documents

• Generic handbook for general centre information

• Subject specific information on student’s particular HND

• Standard presentations

• Induction checklists

• Feedback from candidates

Appeals process

• Difference between Appeal against assessment decision

and a complaint

• Two separate processes

• Appeals are healthy in a centre – why?

Appeals Process

• Appeal is a three stage process

- stage 1 – speak to assessor – how recorded?

- stage 2 – refer to internal verifier

- stage 3 – independent party/appeals panel

within centre

• Recording documentation

Malpractice

• Very important to have transparent, clear policies and

procedures in place in relation to malpractice

• Two types of malpractice

- student malpractice

- centre malpractice involving staff

Malpractice - Student

• Provide student with clear definitions of what constitutes

malpractice in plain language they understand

• Explain investigation process should any student

malpractice be suspected

• Provide details of possible consequences to student if they

are found to be involved in any malpractice activity –

induction handbook

Malpractice - Centre

• Provide staff with clear guidelines on what constitutes

malpractice in plain language they understand

• Explain investigation process should any member of staff

be suspected of being involved in malpractice act.

• Provide details of possible consequences to staff if they are

found to be involved in any malpractice activity

Data Processes

• Important to have clearly documented procedures to

ensure

- accurate and complete data is sent to SQA

- data is sent in to meet published deadlines

- proper recording documentation is in place

and is fully completed and signed off

- once units are internally verified submit

results to SQA (part of your plan)

- audit trail in place

Questions