+ All Categories
Home > Documents > Develop Competency-Based Assessment Plans

Develop Competency-Based Assessment Plans

Date post: 27-Feb-2018
Category:
Upload: thehrscientist
View: 218 times
Download: 0 times
Share this document with a friend

of 52

Transcript
  • 7/25/2019 Develop Competency-Based Assessment Plans

    1/52

    - 1 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Develop Competency-Based Assessment Plans

  • 7/25/2019 Develop Competency-Based Assessment Plans

    2/52

    - 2 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Table of Contents

    Message from the Director of Quality Assurance Division ............................................................. 5

    Learner Profile ............................................................................................................................. 6

    Reflection on Learner Profiles....................................................................................................... 7

    Will Technology-assisted, Work-based, Continuous Assessments Work? ......................................... 8

    Using Social Media-based E-Portfolios in Competency-based Assessment ....................................... 9

    What is Competency? ................................................................................................................ 11

    Packaging of Competency Standard (CS) and Curriculum, Training and Assessment Guide (CTAG) 11

    Competency Standard Template ................................................................................................. 12

    Key Changes to Competency Standard Template ........................................................................ 13

    Revised Competency Standard Template .................................................................................... 14

    Curriculum, Training and Assessment Guide (CTAG) .................................................................... 14

    Curriculum, Training and Assessment Guide (CTAG) Components ................................................ 15

    Key Changes to CTAG Template .................................................................................................. 15

    Competency-based Assessment ................................................................................................. 17

    Requirements of Competency-based Assessment ....................................................................... 17

    Types of Assessment .................................................................................................................. 18

    Commonly Used Assessment Methods ....................................................................................... 19

    Knowledge-based Assessment Methods .......................................................................................... 19

    1. Multiple-Choice Questions (MCQ) ........................................................................................ 19

    2. Written Assessment (Short Answers) ................................................................................... 19

    3. Projects ................................................................................................................................. 20

    4. Portfolio ................................................................................................................................ 20

    5. Written Assessment (Reports) .............................................................................................. 21

    6. Oral Questions ...................................................................................................................... 21

    Performance-based Methods ........................................................................................................... 22

    7. Workplace Performance ....................................................................................................... 22

    8. Role-Play / Simulation ........................................................................................................... 22

    Attitudinal Evaluation Methods ........................................................................................................ 23

    9. Observations ......................................................................................................................... 23

  • 7/25/2019 Develop Competency-Based Assessment Plans

    3/52

    - 3 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Principles of Assessment (POA) .................................................................................................. 24

    Rules of Evidence (ROE) ............................................................................................................. 24

    Adult Educator (AE) Qualifications Requirements and Timelines for WSQ Approved Training

    Organisations (ATO) ................................................................................................................... 25

    How do These Changes to Competency Standard (CS) and Curriculum, Training and Assessment

    Guide (CTAG) Affect my Assessment Plan Design? ...................................................................... 26

    What is an Assessment Plan? ..................................................................................................... 27

    Components of Assessment Plans .............................................................................................. 27

    WSQ Requirements in Relation to Assessment Plans ................................................................... 28

    Common Shortcomings in Assessment Plan Design ..................................................................... 29

    Example 1: Inappropriate Development of Assessment Tools ......................................................... 29

    Example 2: Assessment Criteria do not Address the Performance Statement / Performance Criteria

    Sufficiently ........................................................................................................................................ 30

    Example 3: Incomplete Instructions in Assessment Specifications .................................................. 31

    Example 4: Conflicting Instructions in Assessment Plan ................................................................... 33

    Example 5: Difficulty in Gathering Evidence ..................................................................................... 34

    Example 6: MCQ Questions with Give-away Answers.................................................................. 35

    Other Shortcomings in Assessment Plans ......................................................................................... 36

    Selection of Assessment Methods .............................................................................................. 37

    General Good Practices in Assessment Plan Development ........................................................... 37

    Assessment Plan Validation ........................................................................................................ 38

    Assessment Plan Validation Checklist ......................................................................................... 39

    POA / ROE Validation Checklist .................................................................................................. 40

    WSQ Curriculum Development and Pre-Accreditation Checklist .................................................. 45

    References ................................................................................................................................. 50

    Useful Websites ......................................................................................................................... 51

    Version Control Record .............................................................................................................. 52

  • 7/25/2019 Develop Competency-Based Assessment Plans

    4/52

    - 4 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    BLANK PAGE

  • 7/25/2019 Develop Competency-Based Assessment Plans

    5/52

    - 5 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Message from the Director of Quality Assurance Division

    Dear WSQ Developers,

    The Quality Assurance Division (QAD) conducteda survey of the WSQ Approved Training

    Organisations (ATO) in March 2012. It was

    found that most ATOs indicated a need to have

    a better understanding of how to develop

    assessment plans. This further supports QADs

    audit observations that ATOs need to improve

    the design of their assessment plans.

    These notes are purposefully designed for

    developers who are involved in the design and

    development of competency-based assessment

    (CBA) for the Singapore Workforce Skills

    Qualifications (WSQ) System. WSQ is a national credentialing system that

    trains, develops, assesses and recognises adult workers for competencies they

    need to stay employable. Thus, assessment is the integral part of WSQ as it is

    used to assess the learning gain and competencies of the workers.

    I hope these notes will be beneficial to your future development of

    competency-based assessment plans.

    All the best in your future developments!

    Sharon Tan (Ms)

    Director

    Quality Assurance Division

  • 7/25/2019 Develop Competency-Based Assessment Plans

    6/52

    - 6 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Learner Profile

    Learner profile is a set of information collected on how learner learns most

    effectively. This information may be collected through observations, surveys,

    questionnaires, application forms, interviews, etc. It may include information

    like:

    Age

    Barriers to learning (e.g. institutional, situational, psychosocial, etc)

    Educational level

    Employability Skills (ES) Level

    Experience (e.g. work experience, life experience, etc)

    Motivations to learn (e.g. job, self-actualisation, etc)

    Orientation to learning (i.e. immediacy of application for problem solving,

    response to societal expectations, etc)

    Readiness to learn

    Self concept (i.e. self-directed, dependent, etc)

    ______________________________________________

    ______________________________________________

    ______________________________________________

    ______________________________________________

    ______________________________________________

  • 7/25/2019 Develop Competency-Based Assessment Plans

    7/52

    - 7 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Reflection on Learner Profiles

    Draw up the profile of learners who attend your programmes. Pick the

    appropriate characteristics.

    Given your learner profile, how do we design assessments to match learner

    needs and strengths?

    Low-Skilled Elderly

    Non-English Speaking

    Physically challenged

    Less confident

    Fearful of training &

    assessments

    Tech-Savvy Gen Y

    24/7 connected

    Multi-tasking

    Easily bored

    Articulate and confident

    PMEs

    Wealth of experience

    High expectations

    Questioning

    Intellectual

    Learner Profile

  • 7/25/2019 Develop Competency-Based Assessment Plans

    8/52

    - 8 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Will Technology-assisted, Work-based, Continuous Assessments

    Work?

    Discuss with your partner on the appropriateness of the three assessment

    approaches for your learners. Tick the relevant boxes if the approaches provide

    valid, sufficient, current and authentic evidence.

    RULES OF EVIDENCE (ROE)

    Evidence

    Criteria

    Approach 1

    (ACTA V5)

    Approach 2

    (WTP)

    Approach 3

    (DACE)Comments

    Validity of

    evidence

    Sufficiency of

    evidence

    Currency of

    evidence

    Authenticity of

    evidence

    Characteristics

    of Each

    Approach

    Continuous

    assessment

    Blogs

    In-class

    performance

    Workplace

    evidence

    Live

    observation

    Video clip of

    presentation

    Skype to

    cover gaps

    Possible Takeaway

    I can apply the following ideas to modify the practice in my organisation:

  • 7/25/2019 Develop Competency-Based Assessment Plans

    9/52

    - 9 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Using Social Media-based E-Portfolios in Competency-based

    AssessmentA pilot project by Institute for Adult Learning (Lee Wee Chee)

    Background

    IAL recently conducted a pilot run of a new design for four modules of

    Advanced Certificate in Training and Assessment (ACTA). One of the new

    methods that were tested was the use of social-media based e-portfolios for

    continuous, formative and summative assessment.

    Assessment Process

    Learners started collecting evidence (artefacts) of learning and competency

    from the start of the course. These took the form of pictures, reflective blog

    entries capturing their thoughts on the application of learning, and videos of

    them practising their delivery of training and assessment. Using a blog host

    (Wordpress) and video host (YouTube), learners published their entries

    frequently, allowing assessors, trainers and fellow learners to give feedback on

    performance, comment on entries, and identify gaps to cover.

    An Alternative Expression of Competency-based Assessment

    This project built social constructivism, reflection and personalised

    contextualisation into competency-based assessment, allowing assessors to

    have a holistic, developmental view of the learners journey towards

    competence.

    Assessment Framework

    Based on a portfolio assessment framework developed by Cleveland Clinic

    Lerner College of Medicine for medical students1, IAL adapted it to create a

    similar framework that includes e-portfolio and social media elements.

  • 7/25/2019 Develop Competency-Based Assessment Plans

    10/52

    - 10 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Evidence Gathering Tools

    Smartphones, tablets

    With photo-taking and

    video-capturing

    capability

    Evidence Portfolio

    e-Portfolio using social

    media like blog hosts

    (e.g. Wordpress) and

    video hosts (e.g.

    YouTube)

    1Dannefer, E. F., and Henson, L. C. (2007). The portfolio approach to competency-based assessment at the Cleveland Clinic

    Lerner College of Medicine.Academic medicine, 82(5), 493-501.

    SummativeAssessment

    Learners meet withassessor to review allevidence for decision

    on competence

    Formative Assessment

    Learners meet with assessorto review evidence presented

    so far to determineperformance gaps and

    actions to take to close thegaps

    Informal Feedback

    Trainers, assessors and fellow learnersgive feedback on blog entries for

    developmental purposes

    Evidence Database

    Application to own workplace practice: Reflective entrieson applicability of learning to workplace

    Teaching and assessment skills: Videos of skills practicesin class or from the workplace + reflection

    Construction of knowledge: Photographs and entries ofown learning + reflection

    Professional development: Reflective entries onprofessional practice based on self and peer feedback

  • 7/25/2019 Develop Competency-Based Assessment Plans

    11/52

    - 11 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    What is Competency?

    A competency is a measurable set of knowledge, skills, and attitudes that a

    person needs to perform a task effectively

    A competency is not an entire job

    In most cases, several competencies are required for a job

    As competency is task-based, the person may need to transfer the

    competencies to new situations and environment

    Packaging of Competency Standard (CS) and Curriculum, Training and

    Assessment Guide (CTAG)

    1stGeneration CS 2ndGeneration CS 3rdGeneration CS

  • 7/25/2019 Develop Competency-Based Assessment Plans

    12/52

    - 12 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Competency Standard Template

  • 7/25/2019 Develop Competency-Based Assessment Plans

    13/52

    - 13 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Key Changes to Competency Standard Template

    Key CS Components Status of Revision

    Cover page Retain

    Relevant Job

    role(s)/occupation(s)Retain

    Performance Statements /

    Competency Element /

    Performance Criteria

    Retain and Enhance

    Assumed Skills and Knowledge Retain and Enhance

    Underpinning Knowledge Retain and Enhance

    Range and Context Retain

    Evidence Sources Retain

  • 7/25/2019 Develop Competency-Based Assessment Plans

    14/52

    - 14 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Revised Competency Standard Template

    Curriculum, Training and Assessment Guide (CTAG)

  • 7/25/2019 Develop Competency-Based Assessment Plans

    15/52

    - 15 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Curriculum, Training and Assessment Guide (CTAG) Components

    Key Changes to CTAG Template

  • 7/25/2019 Develop Competency-Based Assessment Plans

    16/52

    - 16 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Key Changes to CTAG Template

  • 7/25/2019 Develop Competency-Based Assessment Plans

    17/52

    - 17 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Competency-based Assessment

    A collection of evidence to demonstrate that a learner can perform or

    behave according to specific standards

    A form of assessment that is derived from the specification of a set of

    outcomes articulated in the competency standard(s)

    The assessment outcome that will allow the assessor to make judgment

    with respect to the achievement or non-achievement of these outcomes

    (i.e. competent / not yet competent)

    Requirements of Competency-based Assessment Focus on outcomesThe assessment must allow for clear demonstration

    of outcomes of learnersperformance. Evidence must be collected to show

    that a candidate has met every single performance criterion

    Competent/Not yet competent judgement onlyOnly two judgments can

    be made

    Assessment in workplace context Performance must be demonstrated

    and assessed under conditions as close as possible to those under which it

    would normally be practiced.

  • 7/25/2019 Develop Competency-Based Assessment Plans

    18/52

    - 18 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Types of Assessment

    Diagnostic

    o Prior to learning, e.g. training needs assessment

    Formative

    o During learning, to gauge the learning progress and achievement of

    learners for learning (Assessment for learning) does not contribute to

    final assessment result

    Continuous

    o During learning, to gauge the learning acquired for competence/ grading

    (Assessment of learning)contributes to final assessment result

    Summative

    o End of learning, to determine competence

    Skills Recognition

    o Recognition of Prior Learning (RPL)

  • 7/25/2019 Develop Competency-Based Assessment Plans

    19/52

    - 19 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Commonly Used Assessment Methods

    Frequently used competency-based assessment methods may be categorised

    into four broad groupings:

    Knowledge-based Assessment Methods

    1.Multiple-Choice Questions (MCQ)

    Multiple-choice questions consist of a question, followed by a number of

    options. One of the options is the correct answer. The other options, which

    are incorrect, are called distracters.

    Advantages Disadvantages

    As there is minimal writing, a

    substantial amount of content can be

    assessed in a relatively short time

    Can be administered to a group of

    learners concurrently

    Scoring is highly objective (i.e.

    number of correct answers only)

    Validity of evidence blurred by the 25%

    (assuming 4-option MCQ) chance of

    random success.

    Relatively difficult to design good MCQ

    questions - requires skilled designers

    and pilot- testing.

    Method is usually constrained to being a

    supplementary method to test

    knowledge and comprehension.

    Validity decreases with frequent use

    and exposure of MCQ to learners

    2.Written Assessment (Short Answers)

    Short answers require candidates to articulate understanding in written

    form but the answers are relatively short as compared to essays and hence

    may be appropriate for candidates with literacy issues.

    Advantages Disadvantage

    Easy to mark/grade as the answers

    are predictable.

    Answers are limited hence writing

    ability is not unduly stressed.

    Suitable for learners who do not have

    high literacy levels.

    Cannot be used alone and has to be

    supplemented with other methods.

  • 7/25/2019 Develop Competency-Based Assessment Plans

    20/52

    - 20 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    3.Projects

    Projects involve an assessment of a combination of subject knowledge,

    process skills and transferable skills. Projects require learners to apply their

    learning, encourage pro-activity and need an extended duration for

    completion. The outcomes of a project may be presented through differentforms such as a report, portfolio, and/or presentation.

    Advantages Disadvantages

    More flexible (in terms of scope,

    medium for presentation) than other

    assessment methods

    High validity as it requires the

    application of skills and knowledge ina given / real context

    Time consuming to mark

    Challenging to determine learners true

    contributions and correspondingly his

    level of competencies

    Should be complemented with other

    supplementary methods to ensure

    authenticity of evidence

    4.Portfolio

    A portfolio refers to a collection of evidence assembled by the learner to

    demonstrate competence. The main purpose of using a portfolio would be

    the collection of evidence to establish that the learner has demonstrated aset of prescribed performance criteria. A key point to note is that evidence

    that was accumulated many years ago may not be current and has to be in

    line with current practices and standards.

    Advantages Disadvantages

    Flexible as learners take the initiative

    to decide on how the portfolio is to

    be presented and organised.

    Valid form of assessment as it is

    based on real task rather than a

    series of educational tasks.

    Challenging to some learners due to the

    flexibility

    May be difficult for assessors to map

    the portfolio to assessment criteria and

    require a face-to-face interview to

    ascertain competence

  • 7/25/2019 Develop Competency-Based Assessment Plans

    21/52

    - 21 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    5.Written Assessment (Reports)

    A report is a form of extended written assignment whereby information and

    often recommendations for action are presented.

    Advantage Disadvantage

    Relevance to realistic context hence

    more practical than an academic

    essay

    Quite demanding as learners may be

    required to analyse on what they have

    done and draw conclusions

    6.Oral Questions

    Oral Questioning require learners to articulate their understanding through

    a verbal face-to-face question and answer (Q&A) process. This method isuseful when addressing a diverse group of learners with varying literacy

    levels as the assessor is able to paraphrase questions to help the candidates

    in understanding.

    Advantage Disadvantage

    Able to address diverse needs of

    candidates

    Challenging for inexperience assessors

    as they will have to be able to think on

    their feet and paraphrase the questionswithout providing undue stress to the

    candidate

  • 7/25/2019 Develop Competency-Based Assessment Plans

    22/52

    - 22 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Performance-based Methods

    7.Workplace Performance

    This is arguably the best method to collect reliable and authentic evidence

    of a learnersactual competencies. This is because the assessment context

    is set under actual work conditions that require demonstration of actualoccupational competencies and maximises the degree of realism in the

    assessment process.

    Advantages Disadvantages

    Good indicator of a learners actual

    work performance

    High authenticity and reliability of

    evidence

    Constrained by opportunities available

    to observe performance for all range of

    activities specified under standards

    Requires considerable time andresources to assess all the performance

    criteria

    8.Role-Play / Simulation

    Role-playing or simulation provides an alternative for workplace

    performance. It can be seen as an orchestrated scenario that seeks togather evidence on a learnerscompetencies.

    Advantages Disadvantages

    Allows control of different settings

    required to assess a full range of

    activities stated under standards

    Flexibility in time allocation

    Allows standardization of activities toenhance fairness of assessment

    Detached from realistic workplace

    conditions

    Requires considerable investment in

    facilities and equipment to simulate

    actual work setting

  • 7/25/2019 Develop Competency-Based Assessment Plans

    23/52

    - 23 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Attitudinal Evaluation Methods

    9.Observations

    This method involves observing the behaviour of learners, recording as

    much as possible, or whatever appears to be useful, important, or unusual.

    Once the data are recorded, the observers task is to make sense and draw

    conclusions from it. Observation requires preparation time. It is necessary

    to determine why you are observing, what you expect to see or think you

    might see and how you will record what you see.

    Attitudinal evaluation requires determination of a proxy behaviour that

    best presents the attitude item / component.

    The selection and determination of the proxy behaviour requires researchbacking and testing. And there has to be a relationship between the proxy

    behaviour and attitude, before it can be safely used for attitudinal

    evaluation.

    Advantages Disadvantages

    Structured observation allows the

    observer to focus on specific

    behaviour that indicate positive andnegative feelings or attitudes

    Structured observation allows for

    recording of reliable and relevant

    data

    Structured observation incurs the risk of

    the observer overlooking concurrent

    relevant events

    In the case of third party observations

    where inputs from direct supervisors

    are sought, key issue is the degree of

    objectivity and reliability of the

    evaluation

  • 7/25/2019 Develop Competency-Based Assessment Plans

    24/52

    - 24 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Principles of Assessment (POA)

    Validityrefers to the strength of assessment tasks and criteria to accurately

    measure learners attainment of the intended competencies level (i.e. how

    well the test really reflects the knowledge, skills and attitudes of the

    Competency Standard). It includes content validity, construct validity, facevalidity and predictive validity

    Reliability refers to the consistency of measured outcomes via clear and

    consistent processes for setting, marking, grading etc when a test (all

    factors being equal) is repeated on a population of individuals or groups.

    Flexibilityrefers to the consideration of the various needs of learners and

    allows assessments to be conducted either on or off the job at mutually

    convenient times and situations.

    Fairnessrefers to the equality in the conduct of assessment and does not

    advantage or disadvantage particular learners (e.g. amount of assessed

    work should be manageable).

    Rules of Evidence (ROE)

    ValidEvidence

    o Meets the Competency Standards and assessment criteria

    SufficientEvidence

    o Sufficient evidence to show competency for all Performance

    Statements / Performance Criteria and Underpinning Knowledge

    CurrentEvidence

    o Evidence collected is based on current situation and not outdated

    AuthenticEvidence

    o Learners own work

  • 7/25/2019 Develop Competency-Based Assessment Plans

    25/52

    - 25 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Adult Educator (AE) Qualifications Requirements and Timelines for

    WSQ Approved Training Organisations (ATO)

    In-House WSQ ATOs

    RolePhase I

    By 1 Oct 2012

    Phase II

    By 1 Oct 2013

    Phase III

    By 1 Oct 2014

    Trainer and/or assessor At least 50%with

    Workplace Trainer

    Programme (WTP) or

    equivalent

    At least 65% with

    Workplace Trainer

    Programme (WTP) or

    equivalent

    At least 80%with

    Workplace Trainer

    Programme (WTP) or

    equivalent

    Master Trainer At least 1 master trainer* must attain full ACTA or equivalent by 1 Oct

    2013

    Curriculum Developer At least 1 developer* must attain full ACTA or equivalent by 1 Oct 2013

    Note:

    New ATOs or existing ATOs who are moving into new areas of training are required to meet

    requirements in Phase I with immediate effect.

    WSQ In-house ATOs are companies/organisations that train and/or assess their own staff

    internally.

    Figures are based on declaration during accreditation and deployment

    * Can be the same person

    Public WSQ ATOs

    Role Phase I

    By 1 Oct 2012

    Phase II

    By 1 Oct 2013

    Phase III

    By 1 Oct 2014

    Trainer and/or assessor At least 50%with full

    Advanced Certificate in

    Training and

    Assessment (ACTA) or

    equivalent

    At least 65%with

    Advanced Certificate in

    Training and

    Assessment (ACTA) or

    equivalent

    At least 80%with

    Advanced Certificate in

    Training and

    Assessment (ACTA) or

    equivalent

    Curriculum Developer 100% with Diploma in Adult and Continuing Education (DACE) or

    equivalentby 1 Oct 2015

    Note:

    New ATOs or existing ATOs who are moving into new areas of training are required to meet

    requirements in Phase I with immediate effect. WSQ ATOs are external training providers that are in the business of conducting training and/or

    assessment

    Today, curriculum developers of ATOs are already required to meet ACTA or its equivalent. They

    will also be required to meet the new DACE requirement by 1 Oct 2015.

    Figures are based on declaration during accreditation and deployment . ATOs are required to

    ensure that its actual deployed AEs for each class delivery under each approved framework meet

    the qualifications requirements

    Partial Completion of DACE: Individuals who have already attained DACE competency units (CU)

    C2, C3, C5, C6 and C7, may function as WSQ Trainers and Assessors at WSQ ATOs in lieu of

    completing the ACTA Qualification. Such individuals are encouraged to complete the remaining

    units of the DACE qualification as the full DACE qualification will enable them to practice as aWSQ Curriculum Developer at WSQ ATOs.

  • 7/25/2019 Develop Competency-Based Assessment Plans

    26/52

    - 26 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    How do These Changes to Competency Standard (CS) and Curriculum,

    Training and Assessment Guide (CTAG) Affect my Assessment Plan

    Design?

    For Competency Standards (CS)

    Examine the assumed knowledge, skills and attitudes (KSA)

    Examine the level of Blooms taxonomy of cognitive, affective and

    psychomotor domains used in the Performance Statements / Performance

    Criteria and Underpinning Knowledge

    Examine the evidence source requirements

    For Curriculum, Training and Assessment Guide (CTAG)

    Review the industry-adopted strategies and hours

    Mandatory / legal requirements and relevant training and assessment

    hours

    Other recommended strategies

    Domain knowledge (technical / vocational qualifications)

    Domain industry experience

    Adult educator qualifications and experience

  • 7/25/2019 Develop Competency-Based Assessment Plans

    27/52

    - 27 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    What is an Assessment Plan?

    An Assessment Plan is a document that provides information on how the

    assessment will be structured, what is involved and the assessment criteria

    that candidates will be assessed against.

    Components of Assessment Plans

  • 7/25/2019 Develop Competency-Based Assessment Plans

    28/52

    - 28 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    WSQ Requirements in Relation to Assessment Plans

    Principles of Assessment (POA) are complied with

    Rules of Evidence (ROE) are complied with

    Mandatory assessment strategies and requirements (if any is specified inthe CTAG)

    All Performance Criteria / Performance Statements and Underpinning

    Knowledge are assessed

    Proposed Assessor-Learner ratio

    Assessor materials are complete, meet needs of assessor

    Assessment Plan is developed with:

    o Clear instructions for preparations for assessment and documentation of

    assessment e.g. set up of equipment, conduct of specific methods, time

    for responses, appeal process, checklists

    o Assessment instructions to guide assessor

    o Assessment tools are provided e.g. role play scripts, answers to written

    tests

    o Assessment records with clear assessment criteria indicated

    o Resources / References (library, web)

    o Glossary provided

    o Version control

  • 7/25/2019 Develop Competency-Based Assessment Plans

    29/52

    - 29 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 1: Inappropriate Development of Assessment Tools

    Description: A role play script was provided. It was crafted in a dialogue as

    follows:

    Problem: This roleplay script does not provide valid evidence of competence as

    the learners only needs to follow the script and the flow of the script. In a real

    workplace situation, the learners will not be given such a script when providing

    customer service. Learners should not be in the know of exactly what to expect

    during each customer service provided.

    General Advice: Role play scripts should be developed appropriately to guide

    and ensure that the learners are adequately assessed through a role play basedon a real workplace situation. Detailed scripts should only be provided to the

    actor (i.e. person not being assessed) to ensure that the role play does not

    deviate from the intended flow of the role play. A scenario may be provided to

    the learner to explain the context of the role play and the instructions to the

    learner should generally cover what he/she will be assessed on during role play.

    A sample role play script developed to assess learner based on a real work

    place situation is as follows:

    Role Play Script (Candidate Copy)

    (Customer walks into the store)

    Customer: Hello.

    Candidate (retailer): Good Afternoon, Mr XXXX. Welcome to OIO Department Store.

    Customer: I am looking for a LCD TV.

    Candidate (retailer): Can you please share with me what kind of TV you are looking for?

    Customer: HD, Cheap, slim, power saving, and good resolution.

    Candidate (retailer): Sure Sir! We have a great range of TV sets that you may like

    (Candidate brings the customer to the TV)

    Candidate retailer : This is one o the latest models o HD TVs that comes with..... .

    Role Play Scenario

    It is a Sunday morning and your department store is going to launch a new product called the

    iRobot. Your organisation anticipates a huge demand for this new electronic item as shown by

    the large sales figures in other countries.

    Role Play Script (Candidates Copy)

    You are the retail assistant at XYZ Company. You turn up for work on time this morning and get

    ready for work. You know that there is a new HOT item to be launched today. You are to

    deliver service excellence to your customer according to your organisational policies and

    procedures.

    You will be assessed on the competencies in Deliver Service Excellence, i.e. Use organisational service culture and values to guide service delivery

  • 7/25/2019 Develop Competency-Based Assessment Plans

    30/52

    - 30 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 2: Assessment Criteria do not Address the Performance

    Statement / Performance Criteria Sufficiently

    Description:Assessment criteria were set differently as compared to thePerformance Criteria in the Competency Standard

    Problem: The Performance Criteria state that it is to ascertain the competency

    of the learner to review team and organisational business plans with the

    objective of identifying new and emerging skill requirements.

    The question set only assesses the individual knowledge on how to evaluate

    business plan and how to identify skill requirements. It does not assess how the

    individual reviewsteam and organisational business plans with the objective of

    identifying new and emerging skill requirements

    General Advice: Be very clear about what the Performance Criteria expect as

    Performance Criteria and Performance Statements require the learner to

    demonstrate competence holistically, rather than in segments.

    Example:

    Competency Standard: LPM-DEV-301C-0 Encourage People

    Performance Criteria: Review team and organisational business plans to identify new

    and emerging skill requirements

    Assessment Method chosen:Written Assignment

    Question set for this PC: Complete a 100-word essay on evaluating business plans and

    how to identify emerging skill requirements

  • 7/25/2019 Develop Competency-Based Assessment Plans

    31/52

    - 31 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 3: Incomplete Instructions in Assessment Specifications

    Description: The ATO only submitted the following table as part of the

    assessment specifications. The rest of the assessment plan consists of thematrix of assessment methods, observation checklist and a role play script.

    S/N Specifications Guidelines

    1 Assessment Method Role Play

    2 Performance Criteria PC 1.1, 1.2, 2.3, 2.4, 2.5, 2.6, 3.1, 3.2, 3.3

    3 Underpinning Knowledge All stated UK

    4 Duration 20 minutes

    5 Assessment Venue Training Centre or Office

    6 Assessment Resources Assessment Checklist

    Assessment Questions and Answers

    7 Assessment Set-up Requirements To be conducted in quiet room with proper tablesand chairs

    8 Conduct of Assessment Conduct Interview

    9 Recording the Assessment Result Record findings and conclusions in theIndividual Assessment Record.

    Record reasons for NYC in remarks column.

    10 Candidate Summary and Feedback Record feedback and findings into the SummaryAssessment Record.

    11 Required Assessment Tools Course Assessment Booklet (refer to Annex A)

    Problem: The specifications do not provide complete information on the

    following:

    Assessment process

    Decision making on assessment outcome for the method and for the whole

    assessment Preparation for assessment (by the assessor and / or assessment centre)

    General Advice: Developer needs to ensure that the instructions and

    specifications are detailed enough to clearly cover the following (not limited

    to):

    Assessment duration

    Assessment venue

    Assessment set up requirements

    Assessment tools

  • 7/25/2019 Develop Competency-Based Assessment Plans

    32/52

    - 32 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Assessor to candidate ratio

    Conduct / provision of feedback

    Conduct of assessment

    Decision making on assessment outcome for the method and for the whole

    assessment

    Preparation for assessment (by the assessor and / or assessment centre)

    Performance Statements / Performance Criteria and Underpinning

    Knowledge covered

    Special instructions (e.g. safety precautions, etc)

  • 7/25/2019 Develop Competency-Based Assessment Plans

    33/52

    - 33 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 4: Conflicting Instructions in Assessment Plan

    Description:Assessment criteria and assessment specifications provided by the

    ATO were not reflective of information (e.g. assessment methods selected foreach Performance Criteria / Performance Statement) provided in the Evidence

    Gathering Plan. Refer to the illustration below.

    Problem: Users of the assessment plan may get confused as to what is the

    correct assessment method/s and whether the assessment plan is of the correct

    version. This may reduce the confidence of the user of the assessment tools and

    create a sense of doubt in the assessment plan being used.

    General Advice: Ensure that all sections in the assessment plan are consistentand do not conflict with each other. Always do a check on whether there is any

    information that may confuse the user.

    EVIDENCE GATHERING PLAN

    ASSESSMENT CHECKLIST

    Performance Criteria andUnderpinning Knowledge

    Assessment Criteria

    Please Tick

    Remarks

    C NYC

    1.2 Communicate the performancemanagement system to employeesand their roles and responsibilitiesin performance management.

    OQ13: Explain how to communicate the performance management

    system to employees and their roles and responsibilities inperformance management.

    Suggested Answers:Candidate is able to explain how to communicate the performancemanagement system to employees and their roles and responsibilitiesin performance management, which may include:

    As part of the performance appraisal

    .............................

    .............................

    Performance Criteria Evidence Gathering Plan Method

    Assessment Tools

    Assessment

    c

    hecklist

    Instructionsto

    Ca

    ndidates

    Instructionsto

    Assessors

    Learningjournal

    Role

    Playscripts

    1.2 Communicate theperformance managementsystem to employees andtheir roles andresponsibilities inperformance management.

    Assessment Criteria:Ability to communicate theimportance of performancemanagement and the rolesand responsibilities tostakeholders

    Work Activity:Able to verbally communicate and adaptcommunication methods of the roles andresponsibilities of performance management toline managers and employees

    Role PlayAA1

    The assessment

    method chosen

    here is role play

    The assessment

    checklist states that it

    is an oral question

  • 7/25/2019 Develop Competency-Based Assessment Plans

    34/52

    - 34 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 5: Difficulty in Gathering Evidence

    Description: Assessment specifications require the assessor to observe more

    than two concurrent practical performances for assessment.

    Problem: In order for the assessor to observe any forms of practical

    performance reliably, the assessor needs to be very clear of the context and the

    required role of the candidate. Having to observe more than two different

    practical performances is even more challenging, more so for an assessor that

    is not very familiar with the process.

    General Advice: Unless there are strict criteria for assessors, and specific

    consequential actions (i.e. achievement of specific outcome from only onespecific process) being observed, it is advisable for assessment of practical

    performance to be conducted on a one-to-one basis.

    6 Assessment Set-

    up Requirements

    A fully set-up restaurant. This includes providing all the

    relevant items found in a typical restaurant.

    7 Conducting the

    Role Play

    1. The assessment is to be conducted at the restaurant

    (actual workplace)

    2. Three candidates will take on the role of waiter and serve

    customers according to the organisational policies and

    procedures.

    3. Three other people will take on the role of an angry

    customer, an overly excited guest and an indecisive guest.

    4. The role play begins with a guest entering and randomly

    selecting a waiter to serve him / her.

    5. It will end when all the guests have paid the bill and leftthe restaurant

    6. In the event that competency is not being able to be

    observed during the role play, the assessor could use oral

    questioning to ascertain the competency.

    7. The candidate is required to complete the entire role play

    even if the assessor had rated the candidate as Not Yet

    Competent. The result of the assessment should only be

    made known to the candidate at the end of the

    assessment.

  • 7/25/2019 Develop Competency-Based Assessment Plans

    35/52

    - 35 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Example 6: MCQ Questions with Give-awayAnswers

    Description:Although the multiple-choice question may have addressed the UK

    / PC, it has been set such that the answer is very obvious. (See below forexample)

    Problem: If the correct answer is too obvious, it may result in an unreliable

    result as it is unclear if the learner can actually answer the question or if the

    other choices (i.e. distracters) were plausible.

    General Advice: Ensure that the developer of the assessment plan is clear and

    well-qualified in developing a reliable MCQ Questionnaire. This will help ensure

    that the assessment is in line with the rules of evidence. Some general tips fordesigning MCQs include:

    Measure important outcomes only

    Write a clear question

    Ensure that the grammar of the term matches that of the options

    Avoid giving verbal clues in the question

    Provide plausible distracters Avoid all of the above and none of the above

    Vary the length of the correct answer

    Vary the position of the correct answer

    Avoid making the correct answer less definite

    Example:

    1. What is an attribute of a manager? (choose the one correct answer)

    a. Able to communicate ideas to supervisors for implementation

    b. Able to escape from all responsibilities

    c. Able to change the world

    d. Able to punish staff as and when he likes

  • 7/25/2019 Develop Competency-Based Assessment Plans

    36/52

    - 36 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Common Shortcomings in Assessment Plan Design

    Other Shortcomings in Assessment Plans

    Assessment criteria are not specific (e.g. copied directly from Performance

    Criteria / Performance Statements without elaboration)

    Did no address all the Performance Criteria / Performance Statements or

    Underpinning Knowledge

    Did not indicate the assessment coverage in assessment plan

    Inclusion of other instructions that do not relate to the assessment (e.g.

    payment policy, refund policy, etc)

    Lack of assessment tools (e.g. oral questions and suggested answers, etc)

    Lack of instructions to assessors on

    o conduct of assessments

    o usage of assessment tools

    Lack of instructions to learners

    ______________________________________________________________

    ______________________________________________________________

  • 7/25/2019 Develop Competency-Based Assessment Plans

    37/52

    - 37 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Selection of Assessment Methods

    General Good Practices in Assessment Plan Development

    Maintaining an assessment tools bank (i.e. different assessment methods

    selected systematically to ensure reliability of assessments conducted)

    Conducting regular validation sessions to ensure currency of assessment

    plans

    Collecting feedback from various parties (e.g. assessors, trainers,

    developers, participants, observers, etc) on the usefulness of the

    assessment

    Conducting pilot tests for assessment plans before making adjustments

    ______________________________________________________________

    ______________________________________________________________

  • 7/25/2019 Develop Competency-Based Assessment Plans

    38/52

    - 38 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Assessment Plan Validation

    Purpose of validation

    o To determine if assessment plan meets the Principles of Assessment and

    Rules of Evidence, learners profile, and contextualisation (organisation,

    industry and legal) and learnersrequirements

    Who are the stakeholders

    o Line managers/supervisors, training managers, subject matter experts

    (SME), trainers, assessors

    What to focus for validation

    o Assessment criteria and evidence requirements

    o Assessment environment

    o

    Assessment methods and toolso Principles of Assessment and Rules of Evidence

    o Practicality

    o Clarity of instructions

    o Cost effectiveness

    o Contextualisation (e.g. organisation, industry and legal) and learner

    requirements

    When should validation take place

    o Before assessment

    o During assessment

    o After assessment

    Follow up to validation

  • 7/25/2019 Develop Competency-Based Assessment Plans

    39/52

    - 39 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Assessment Plan Validation Checklist

    Purpose of Validation:

    Stakeholders involved:

    Areas of Focus:

    Assessment criteria and evidence requirements

    Assessment environment

    Assessment methods and tools

    Principles of Assessment / Rules of Evidence*

    Practicality

    Clarity of instructions

    Cost effectiveness

    Contextualisation (organisation, industry and legal) and learnersrequirements

    ______________________________________________________

    ______________________________________________________

    Assessment Venue:

    Follow up:

    *Refer to POA / ROE Validation Checklist

  • 7/25/2019 Develop Competency-Based Assessment Plans

    40/52

    - 40 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    POA / ROE Validation Checklist

    Underlying PrincipleSuggested areas to look into

    (not limited to)

    Compliance Mitigating Factor

    (if non-

    compliant)Yes No

    Principle of Assessment - Validity

    Assessment activities are

    based on actualworkplace practices and

    contexts

    Assessment criteria

    Assessment specifications Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Assessment activities

    relates directly address

    the PS / PERFORMANCE

    CRITERIA and UKs

    Assessment tools: case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etcAssessment tools address

    the candidates ability to

    meet the level of

    performance required by

    the Competency Standard

    Assessment tools: case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Assessment activities

    planned are holistic and

    integrated

    Assessment methods

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessmentchecklists, observation

    checklists, etc

    The assessment method

    and tools developed are

    appropriate for the

    assessment system

    specified by industry

    (where relevant)

    Assessment methods

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    All information in theassessment Plan has been

    validated by an

    appropriate person (e.g.

    master trainer, subject

    matter expert, etc) with

    expertise in the

    competency/is being

    assessed

    Developer / co-developer Validation panel

    Subject matter expert

  • 7/25/2019 Develop Competency-Based Assessment Plans

    41/52

    - 41 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Underlying PrincipleSuggested areas to look into

    (not limited to)

    Compliance Mitigating Factor

    (if non-

    compliant)Yes No

    Principle of Assessment - Reliability

    Critical elements have

    been identified and

    sampling used to ensure

    that the most importantaspects are assessed.

    Assessment specifications

    Assessment tools: case study

    scenarios, role play scripts,

    scenarios, assessmentchecklists, observation

    checklists, etc

    Clear assessment criteria

    have been provided for

    the assessor in the

    assessment / observation

    checklists (e.g. clear guide

    to assess and individual

    as C or NYC)

    Assessment / Observation

    Checklist/s

    Clear instructions have

    been provided to ensurethat assessors conduct

    the assessment activity

    consistently and make

    consistent decisions over

    time and with different

    candidates

    Instructions to assessors

    Assessment specifications Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Where multiple assessors

    are involved in

    conducting parallel

    assessment events,

    assessment strategiesand approaches are

    consistent

    Assessment methods

    Assessment criteria

    Assessment tools: case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Consistent instructions

    are provided to learner/s

    and procedures for

    undertaking the

    assessment

    Instructions to candidates

    Where work samples are

    used, learners are

    provided with clear

    guidelines on

    requirements relating to

    authenticity and currency

    of submissions

    Instructions to assessors /

    candidates

    Assessment specifications

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Where applicable,

    assessment specifications

    and tools are able to

    produce consistent

    assessment outcomes

    conducted in different

    contexts

    Assessment methods

    Instructions to assessors /

    candidates

    Assessment criteria

    Other assessment tools (whereapplicable): case study

  • 7/25/2019 Develop Competency-Based Assessment Plans

    42/52

    - 42 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Underlying PrincipleSuggested areas to look into

    (not limited to)

    Compliance Mitigating Factor

    (if non-

    compliant)Yes No

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Principle of Assessment - FlexibilityThe assessment approach

    can be adapted to meet

    the needs of all

    candidates, workplaces

    and/or stakeholders.

    Assessment methods

    Instructions to assessors /

    candidates

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Wherever practical and

    appropriate, assessment

    can be negotiated and

    agreed between the

    assessor and the

    candidate

    Instructions to assessors /

    candidates

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    The assessment strategy

    adequately covers both

    on and off the job

    components of the

    training

    Assessment specifications

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Policies, procedures,

    guidelines and standards

    are put in place to

    support assessors in the

    consistent application of

    reasonable adjustment/s

    where required

    Instructions to assessors /

    candidates

    Assessment specifications

    Principle of Assessment - Fairness

    Candidates are given

    clear and timely

    information on

    assessment and what is

    required.

    Instructions to assessors /

    candidates

    Assessment specifications

    Information provided to

    candidate/s covers:

    the assessment

    method/s assessment process

    Instructions to candidates

  • 7/25/2019 Develop Competency-Based Assessment Plans

    43/52

    - 43 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Underlying PrincipleSuggested areas to look into

    (not limited to)

    Compliance Mitigating Factor

    (if non-

    compliant)Yes No

    the criteria against

    which they are being

    assessed

    when and how they

    will receive feedback appeal process

    Assessment strategy

    caters for and supports

    the language, literacy and

    numeracy needs of all

    candidates in accordance

    with the assumed skills

    and knowledge provided

    in the CTAG (where

    provided)

    Assessment methods

    Instructions to assessors /

    candidates

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etcSpecial needs (e.g.;

    geographical, financial,

    social) of the candidate/s

    have been considered in

    the development and

    conduct of the

    assessment strategy.

    Instructions to assessors /

    candidates

    Assessment specifications

    Reasonable adjustment

    can be made to the

    assessment strategy to

    ensure equity for thecandidate/s, while

    maintaining the integrity

    of the outcomes.

    Instructions to assessors /

    candidates

    Assessment specifications

    Opportunities for

    feedback and review of

    all aspects will be

    provided to candidates.

    Instructions to assessors /

    candidates

    Assessment specifications

    Rule of EvidenceValidity

    Assessment strategy and

    tools accurately address

    all PS / PERFORMANCE

    CRITERIA and UK to prove

    competence of the

    learners

    Assessment specifications

    Assessment criteria Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Rule of EvidenceSufficiency

    Enough / More than

    enough* evidence is

    provided to confirm that

    the learner is competentin meeting all PS /

    Assessment specifications

    Assessment criteria

    Other assessment tools (where

    applicable): case studyscenarios, role play scripts,

  • 7/25/2019 Develop Competency-Based Assessment Plans

    44/52

    - 44 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Underlying PrincipleSuggested areas to look into

    (not limited to)

    Compliance Mitigating Factor

    (if non-

    compliant)Yes No

    PERFORMANCE CRITERIA

    and UK

    * As a general guide,

    there should be multiplesources of evidence (e.g.

    at least two methods per

    assessment plan)

    scenarios, assessment

    checklists, observation

    checklists, etc

    Rule of EvidenceCurrency

    Evidence collected is

    based on current

    situation and is not

    outdated.

    Instructions to assessors /

    candidates

    Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

    Evidence collected is not

    gathered across a period

    that is too long (e.g. six

    months)

    Instructions to assessors /

    candidates

    Assessment specifications

    Rule of EvidenceAuthenticity

    Evidence of competence

    provided by learners are

    produced by the learnerhimself / herself alone

    Instructions to assessors /

    candidates

    Assessment specifications Assessment criteria

    Other assessment tools (where

    applicable): case study

    scenarios, role play scripts,

    scenarios, assessment

    checklists, observation

    checklists, etc

  • 7/25/2019 Develop Competency-Based Assessment Plans

    45/52

    - 45 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    WSQ Curriculum Development and Pre-Accreditation Checklist

    This Checklist serves as a self-assessment tool for the ATO to ascertain if their proposed

    course is likely to meet WSQ course accreditation (CA) requirements.

    It is based on existing CA audit criteria of Course Design and Development, Assessment

    Design, Adult Educator Requirements, and Facilities and Equipment.

    Detailed descriptions of the criteria are provided to enable Curriculum Developers and ATOsto determine if their proposed course meets the expectations of a WSQ course and if not, to

    do the necessary revisions and enhancements before submitting the course to WDA for

    accreditation.

    Aftera course is submitted for accreditation, the ATO will be given up to 10 working days to

    do any necessary revisions and enhancements. Thereafter, the application may be audited

    based on the latest set of documents available to WDA.

    It is therefore important that the ATO conducts a self-check beforesubmitting the course for

    accreditation.

    The Checklist is a reference guide and added resource for Curriculum Developers and

    ATOs. The Checklist does not need to be submitted with the CA application.

    WSQ CA requirements ATOs self-check

    1.1 Course Design and Development

    1.1.1 Meet CEs, PCs/PSs and UK Yes No

    Do the learning outcomes and coverage of content meet all Competency

    Elements, Performance Criteria / Performance Statements andUnderpinning Knowledge?

    Are the Range and Context / Range of Application requirements compliedwith (as specified in the CS and CTAG)?

    1.1.2 Mandatory learning strategies and requirements (if specified in CTAG) Yes No

    Are mandatory learning strategies and requirements, including duration,complied with (if any is specified in the CTAG)?

    1.1.3 Courseware is pedagogically sound Yes No

    Learners Guide and/or supplementary materials for learners are availableand sufficient

    Appropriate learning strategies and instructional methods are used:

    o Is the course pitched at the correct competency level as indicated inthe Competency Standard (Blooms model is recommended)?

    o Is the choice of learning strategy and instructional methods appropriateand aligned to programme outcomes and objectives?

    Aligned to adult learning principles:

    o Are active and learner-centric methods adopted?

  • 7/25/2019 Develop Competency-Based Assessment Plans

    46/52

    - 46 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    o Are activities planned to have variety to sustain the learners interest,engage and motivate them?

    o Are activities planned to incorporate checkpoints for summary, recap oflearning etc to maximise learning retention?

    Tailored to needs of targeted adult learners:

    o Does the course incorporate bite-size learning?

    o Does the course address needs of learners e.g. cultural background,language proficiency?

    o Does the course cater to various learning styles of learners?

    Is the course contextualised to organisation / industrial / legalrequirements?

    Is appropriate media e.g. social media, audio/video used to deliver thetraining?

    Is the time allocated for learning and activities appropriate and sufficient?

    1.1.4 Proposed Trainer-Learner ratio Yes No

    Does the proposed Trainer-Learner ratio meet ratio specified /recommended in CTAG?

    In the absence of a specified / recommended ratio, is the proposed ratioappropriate?

    Note:

    If the proposed ratio exceeds or do not meet specified recommendations, ATOmay be asked to provide justification

    1.1.5 Trainer Materials are complete, meet needs of trainer and comprisethese documents :

    Yes No

    Lesson Plan (with clear indication of topic, duration and equipment)

    Clear instructions for preparations required, timing to pace lessons,conduct of learning activities

    Key presentation points to guide facilitator

    Resources / References (library, web)

    Intellectual Property rights observed e.g. acknowledge sources ofinformation

    Glossary provided

    Version control

    1.2 Assessment Design

    1.2.1 Principles of assessment are observed Yes No

    Is the assessment Valid?

  • 7/25/2019 Develop Competency-Based Assessment Plans

    47/52

    - 47 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    o assesses what it claims to assess through collection of evidence that isrelevant to the activity and demonstrates that the PC/PS/UK have beenmet

    Is the assessment Flexible?

    o allows for either on or off-the-job, at mutually convenient times andsituations, and appropriate to the range of contexts / application

    Is the assessment Fair?

    o does not disadvantage candidates and takes into account thecharacteristics of the candidate being assessed? Equitable to allgroups being assessed

    Is the assessment Reliable?

    o consistency of interpretation of evidence and assessment outcomes

    1.2.2 Rules of evidence are met Yes No

    Is Validityensured?

    o evidence meets the specified criteria of the competency standards

    Is Authenticityensured?

    o evidence produced is/can be/will be verified to be the candidates ownwork

    Is Sufficiencyensured?

    o enough evidence is gathered to show competency against all specifiedcriteria of the standard/s

    Is Currencyensured?

    o Evidence to be gathered are current, relevant and applicable in todayscontext

    1.2.3 Mandatory assessment strategies and requirements (if specified inCTAG)

    Yes No

    Are mandatory assessment strategies and requirements, includingduration, complied with (if any is specified in the CTAG)?

    1.2.4 All PCs/PSs and UK are assessed through : Yes No

    A range of assessment methods

    Assessment method is clearly indicated for each item

    Note:

    ATO may be asked to provide explanations and justifications of theassessment design, methods and tools employed

    1.2.5 Proposed Assessor-Learner ratio Yes No

    Does the proposed Assessor-Learner ratio meet ratio specified /recommended in the CTAG?

  • 7/25/2019 Develop Competency-Based Assessment Plans

    48/52

    - 48 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    In the absence of a specified / recommended ration, is the proposed ratioappropriate?

    Note:

    ATO may be asked to justify and provide explanations if the proposed ratioexceeds or do not meet specified / recommendations

    1.2.6 Assessor Materials are complete, meets needs of assessor, and

    comprises these documents : Yes No

    Clear instructions for preparations for assessment and documentation ofassessment e.g. set up of equipment, conduct of specific methods, timefor responses, appeal process, checklists

    Key assessment pointers to guide assessor

    Assessment tools are provided e.g. role play scripts, answers to writtentests

    Assessment records with clear assessment criteria/grading indicated

    Resources / References (library, web)

    Glossary provided

    Version control

    1.3 Adult Educator (AE) requirements

    1.3.1 Proposed AEs meet the following requirements as specified /recommended in the CTAG

    Yes No

    All proposed developers, trainers, assessors possess adequate:

    o Domain industry experience

    o Domain qualification

    o AE qualifications

    o AE experience

    Note:

    CVs of all AEs to be submitted for reference and verification. ATO may befurther asked to provide copies of relevant certificates/scrolls/transcripts.

    If subject matter experts (SMEs) are used, the SME/s possess adequate:

    o Domain industry experience

    o Domain industry qualification

    Note:

    CVs of all SMEs to be submitted for reference and verification. ATO may befurther asked to provide copies of relevant certificates/scrolls/transcripts.

    ATO is to also note that submission of profiles for SMEs is not sufficient.

    1.4 Facilities and Equipment

  • 7/25/2019 Develop Competency-Based Assessment Plans

    49/52

    - 49 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    1.4.1 Proposed facilities, equipment and other resources Yes No

    Are proposed facilities, equipment and other resources adequate tosupport the effective delivery of training and assessment in accordance tothe requirements as specified in the CTAG?

    1.4.2 Training and assessment site/s comply with all fire, health and safety

    requirements

    Yes No

    Do training and assessment site/s, including its facilities and equipmentcontained within, comply with all legislations and regulations governing theiruse?

    Have all necessary and valid registrations, permits and licenses for theoperation and use of the training and assessment site/s, facilities andequipment been obtained from the proper authorities?

  • 7/25/2019 Develop Competency-Based Assessment Plans

    50/52

    - 50 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    References

    Dannefer, E. F., and Henson, L. C. (2007). The portfolio approach to

    competency-based assessment at the Cleveland Clinic Lerner College of

    Medicine.Academic medicine, 82(5), 493-501.

    Singapore Workforce Development Agency (2007). WSQ Assessment

    Design Guide.Assessment methods1322.

    Institute for Adult Learning (2011),DACE Develop Assessment Tools,

    Learner Guidefrom IAL

    Institute for Adult Learning (2008),CU5 Develop a Competency-Based

    Assessment, Learner Guide from IAL

  • 7/25/2019 Develop Competency-Based Assessment Plans

    51/52

    - 51 -

    Singapore Workforce Development Agency Quality Assurance Division

    Develop Competency-Based Assessment Plans

    Version 1.1 (14 October 2012)

    Useful Websites

    CA self-assessment checklist-

    http://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-

    Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdf

    WSQ terms and conditions-

    https://www.skillsconnect.gov.sg/web/guest/termsAndConditions#

    External Guide for Continuous Improvement Review-

    http://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8

    )4Nov11(A).pdf

    Guidelines for submitting results and printing SOAs-

    http://app2.wda.gov.sg/data/ImgCont/487/OnlineSubmissionForm.zip

    Practical Guide on Managing For Training and Assessment Excellence-

    http://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForu

    mheldon24May07.pdf

    Guide on updating WDA on changes in Course Information

    http://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.

    pdf

    Guide on updating WDA on changes in Organisation Information-

    http://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationP

    rofile.pdf

    Credit exemption guide -http://www.ial.edu.sg/index.aspx?id=81

    http://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdfhttp://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdfhttp://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdfhttps://www.skillsconnect.gov.sg/web/guest/termsAndConditionshttp://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8)4Nov11(A).pdfhttp://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8)4Nov11(A).pdfhttp://app2.wda.gov.sg/data/ImgCont/487/OnlineSubmissionForm.ziphttp://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForumheldon24May07.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForumheldon24May07.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationProfile.pdfhttp://www.ial.edu.sg/index.aspx?id=81http://www.ial.edu.sg/index.aspx?id=81http://www.ial.edu.sg/index.aspx?id=81http://www.ial.edu.sg/index.aspx?id=81http://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateOrganizationProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/ViewandUpdateCourseProfile.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForumheldon24May07.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/WSQPracticalGuideforCDForumheldon24May07.pdfhttp://app2.wda.gov.sg/data/ImgCont/487/OnlineSubmissionForm.ziphttp://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8)4Nov11(A).pdfhttp://app2.wda.gov.sg/data/ImgCont/948/CIR%20External%20Guide(v1.8)4Nov11(A).pdfhttps://www.skillsconnect.gov.sg/web/guest/termsAndConditionshttp://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdfhttp://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdfhttp://app2.wda.gov.sg/data/ImgCont/948/WDA-WSQ-Curriculum%20Development%20&%20Pre-Accreditation%20Checklist(1Aug11)v1.pdf
  • 7/25/2019 Develop Competency-Based Assessment Plans

    52/52

    Version Control Record

    Version Effective Date Changes Author/s

    1.0 27thAugust 2012 Initial version QAD

    1.1 14

    th

    October 2012

    Inclusion of additional notes for

    CD workshop for Public ATO QAD / IAL


Recommended