Competency-based Competency-based Assessment in TQA Assessment in TQA Accredited Senior Accredited Senior Secondary CoursesSecondary Courses

Issues to be discussed in this forum…Issues to be discussed in this forum…

Criterion- & competency-based assessment: how different are they?

Why use competency-based assessment in TQA courses?

Assessment: methods & issues

Quality Assurance: what, why, who and how?

Outcomes-referenced assessment: “Occurs where students’ performances are measured against explicit criteria…its focus [is] on achievement against learning objectives or outcomes.”

(UTAS www.edu.au/tl/supporting/assessment/judgement.html)

Types of outcomes assessment:• Criterion-based approach: Students’ achievement is measured against a range of standards (eg from a ‘C’ to an ‘A’ rating) for each learning outcome

• Competency-based approach (or ‘Mastery learning approach’): Students’ achievement is measured against a predetermined standard (eg ‘pass’/‘fail’) for each learning outcome.

The criterion- and competency-based approaches share the same fundamental

assessment principle.

They are both concerned with assessing learning outcomes against predetermined

standards.

In TQA accredited courses criterion-based assessment has traditionally had three gradations (ratings): ‘C’, ‘B’ and ‘A’ standards.

The recently introduced competency-based assessment courses have a two gradations (ratings): ‘competent’; and ‘not yet competent’.

In theory there is no reason to limit the number of gradations in competency-based assessment – both criterion- and competency-based approaches to assessment are fundamentally the same.

In practice the use of competency-based assessment with two gradations is suited to some kinds of TQA accredited courses.

What kinds of TQA courses use competency-based assessment? (I)

Those with learning outcomes where the focus of assessment is on:

• the learner’s ability to perform an activity (usually

in a defined context)

• the application of knowledge and skills in ‘real world’ contexts.

What kinds of TQA courses use competency-based assessment? (II)Typically these are courses with learning outcomes where it is more important to determine mastery of a discrete learning outcome (ie. the learner ‘can do it’) rather than measuring the degree to which a learner can do something. Underpinning knowledge is implicitly assessed within the context of completion of a task or activity.

Eg. Learning outcome: Use a household fire extinguisher safely

Assessment

Assessment instruments should make clear to the learner:

a) the nature of the specific assessment task (including the

size/format/due date etc)

b) which learning outcomes are being assessed

c) what the learner must demonstrate

d) how the assessment of the task relates to the final

assessment.

Assessment

Examples of common assessment methodologies include:

direct observation verbal/oral testing practical testing written testing written tasks.

Assessment The method chosen should ‘make sense’ within the context of the course and the learning outcomes being assessed. • Direct observation of a practical competency/criterion might be more valid than using a written task.

• Analysis of a product might be more valid when the process is not as significant as the outcome or cannot be easily observed.

Assessment A marking guide should be prepared for each summative assessment instrument. A marking guide articulates ‘what must be demonstrated in the task’. For example:• a ‘correct answer’ list for a test and the number of questions a learner must get right to ‘pass’• a list of the specific features of a product or process that must be demonstrated to ‘pass’.

Assessment

A marking guide should have a direct relationship to the performance indicators described in the course.

The performance indicators may need to be elaborated in order that they ‘make sense’ within a specific assessment task.

Assessment Final assessment decisions should be on-balance, holistic ones based on the evidence of learning outcomes measured against the standards stated in the course.Considerations might include the: relative weighting given to individual assessment tasks consistency of demonstrated achievement 'currency' (evidence from the end of a course may have

greater relevance and reliability than that collected early in the delivery).

Ratings (I)Summative assessment of learning outcomes result in a final rating for each competency. The ‘competent’ rating is a final, overall rating and should not be used as a grade in formative assessment/progress reports.Learners who are assessed as ‘not yet competent’ can be provided with further opportunities to demonstrate competency within the scope of the course’s design-time, provider resources, and TQA reporting requirements.

Ratings (II)

Some providers use 'not yet assessed' in reports on learner's progress (such as term reports).

While this is perfectly acceptable in this context, 'not yet assessed' is NOT a rating that is reportable to the TQA.

Ratings (III)

Providers may use any form of grading they wish in assessment so long as the final rating is reported as per the course document (eg. ‘competent’, ‘not yet competent’, ‘highly competent’).Examples:

% marksmarks out of 10A, B or Cpass/failgold, silver, bronze

AwardsEach TQA accredited course contains a description of award requirements. These are algorithms (or rules) defining the relationship between ratings and a final award.

Eg. To gain an award (‘Pass’) in the course the learner must be assessed as ‘competent’ in all five competencies.

QualificationThe qualification issued by the TQA on the learner’s Qualifications Certificate comprises the course name and the award gained.

Eg. Personal Pathway Planning: Pass

Quality Assurance & the TQA’s Role?

The TQA is responsible for the quality assurance for qualifications it awards in senior secondary courses it has accredited. This means that it has to stand behind the statements made on certificates as to the knowledge and skills of the holder of the certificate.

Audit Model

• How are providers selected for an audit? Risk based approach – ‘flags’ might include: complaints; number of learners/size of provider; major changes in staff; trends in data reported to TQA; time since last audit.

• How do we know if we are/are not being audited this year? TQA notification process.

Audit Model

• Who is audited? Provider not individual

teachers.

• Who does the auditing? TQA staff and

subject experts (when appropriate).

Audit Model

• Is an audit a ‘witch-hunt’? Opportunities for confirmation of best practice, identification of issues, continuous improvement.

• What happens if we ‘fail’ an audit? Audit findings and opportunity to ‘clear’ them, follow-up visits, report to Authority.

What will Auditors look for?

• Evidence that the course has been delivered

• Evidence that learners ‘have learnt’

• Evidence that assessment is based on the

course’s learning outcomes and standards.

Course Delivery

• Planning: did the course provider have an

articulated plan for the delivery of the

course and its assessment?

• Was the course actually delivered?

Planning (I)

A structured course delivery plan would articulate:

• the sequence of course delivery (what parts

of the course were delivered to learners and

in what order)• when formative and summative assessment

occurs.

Planning (II)

A course delivery plan is sometimes called a ‘scope and sequence’.

Course delivery plans could be a ‘whole of course’ plan or could be arranged as a sequence of lesson, weekly, monthly or term plans depending on individual teachers’/providers’ preferred methodology.

Was the Course Actually Delivered?

Evidences could include:

• records of attendance• assessment tasks and student work (especially examples of compulsory tasks such as keeping a journal or log book)

• interviews with past and present students

• records of assessment.

Evidence that Assessment is Based on the Course’s

Learning Outcomes & Standards

Evidences could include:

• summative assessment tasks & marking guides• some examples of student work showing the

application of marking guides

• assessment records and information about how they were used to make final assessment judgements.

Assessment Records Should Contain:• the name of each learner • the competencies assessed and/or the assessment instrument/s used to make the assessment• the date of assessment• the name of the assessor/s.

Assessment Records

The class assessment record could be a single document for a whole class (such as a spreadsheet or paper-based grid table) or comprise a set of records, one for each learner.

The ‘Enhanced’ Audit Model

Used for courses with special risk factor (eg Essential Skills courses with TCE ‘ticks’).

Same as other audit model but auditors will also wish to view examples of the work of ‘borderline’ learners. ‘Borderline’ means the gaining or not the ‘Pass’ award.

Examples of the Work of ‘Borderline’ Learners (I)• The nature of the work - major summative assessment task/s that allows a judgement to be made on all competencies/the award.

• How many? - use those that arise (do not ‘invent’ them). Number of examples will depend on the number of borderline learners & the period that course had been delivered.

Examples of the Work of ‘Borderline’ Learners (II)

• How kept? - archive copies (photocopies, CD-ROM or other storage mechanisms).

• Where kept? - provider responsibility. System needs to be robust enough to handle staff changes.