Slide 1

SEE PROGRAM MODERATION WORKSHOP 2014 LWA 2014 1 LWA is a contracted supplier of assessment validation services, moderation workshops and professional development services under the Skills for Education and Employment Program funded by the Department of Industry

Slide 2

AGENDA 9.00 9.30Registration, on arrival tea and coffee 9.30 9.40Welcome, introductions, overview and purpose of the day 9.40 10.10SESSION 1: IV IN THE SEE 10.10 10.55 SESSION 2: MODERATION PTAs 10.55 11.15Morning tea 11.15 11.35 SESSION 2: CONTINUED Group feedback and discussion of PTA moderation 11.35 12.55 SESSION 3: MODERATION Learning tasks Group feedback and discussion of Learning moderation 12.55 1.30Lunch 1.30 3.00 SESSION 4: MODERATION Oral communication tasks 3.00 3.15Afternoon tea 3.15 3.45 SESSION 4: CONTINUED Group feedback and discussion of Oral Communication moderation 3.45 4.00Wrap up and evaluation forms LWA 2014 2

Slide 3

SESSION 1: IV IN THE SEE Handout 1: What is needed for verification Do you have any questions about what you need to provide? LWA 2014 3

Slide 4

IV REQUIREMENTS - PTA KPI 3 Accurate Assessment Were the assessment tasks appropriate for the ACSF level? PTA kit appropriately aligned to ACSF indicators PTA kit contains a sufficient range of tasks to adequately assess ACSF indicators Is there sufficient evidence that the provider has assessed the client accurately against the ACSF? Student response demonstrates exit competence Sufficient assessor analysis of student performance against the ACSF indicators claimed LWA 2014 4

Slide 5

IV REQUIREMENTS IPA/LOA KPI 3 Accurate Assessment Were the assessment tasks appropriate for the ACSF? Tasks appropriately aligned to the ACSF Task materials are appropriate for use as assessment tasks Is there sufficient evidence that the provider has assessed the client accurately against the ACSF? Student response demonstrates exit competence Sufficient assessor analysis of student performance against the ACSF Range of text types demonstrated KPI 4 Quality Training Are the tasks relevant to the clients learning needs? Tasks are appropriate for the client cohort Tasks are appropriate for adult learners LWA 2014 5

Slide 6

TODAY WE ARE MODERATING, NOT VERIFYING Tasks and client responses stand alone for moderation purposes Consistency in the interpretation and application of the ACSF Build on professional knowledge Support teaching and assessment practices LWA 2014 6

Slide 7

SESSION 2 Moderation Activities: PTAs LWA 2014 7

Slide 8

SESSION 2: PTA TASKS 1, 3, 4 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? LWA 2014 8

Slide 9

PTA 1 :TASK, LEARNING & OC LWA 2014 9 TASK TASK: Aligned Detailed instructions to the assessor Appropriate prompts to the assessor Both learning indicators comprehensively addressed Excellent prompts to the assessor to observe.02 evidence throughout the PTA More familiar questions could be used at the start of the interview, e.g. introductory comments relating to personal details Modified questions may be required for Language clients, e.g. personal details, background

Slide 10

PTA 1: CLIENT RESPONSE, LEARNING LWA 2014 10 2.01 Client Sample: Demonstrated Client identifies some goals and areas of learning need e.g. numeracy, work related skills Evidence of a learning goal and the investigating of relevant learning pathway e.g. has investigated requirements for roadside slashing business Evidence of barriers e.g. family, financial skills Assessor has recorded detailed client quotes 2.02Client Sample: Not demonstrated Evidence more aligned to 1.02 Further questions could have been asked e.g about White Card training, to prompt client to expand on strategies used to gain this qualification Source of information about web search unclear Evidence could have been strengthened if assessor had noted any strategies used by the client during the assessment

Slide 11

PTA 1: CLIENT RESPONSE, OC 3.07 Client Sample: Not demonstrated Evidence suggests potential level 3 but it is not clear for example what are the clients actual words and what has been paraphrased More in depth discussion of some topics, for example, the CFS may have allowed for gathering of stronger evidence at level 3 than a short response for every question e.g. What is involved with being a CFS volunteer? Tell me about the training required for the CFS. What type of person is suited to the CFS? Quotes captured need to demonstrate appropriate levels of complexity Assessor comments need to be supported by reference to specific quotes. The reporting sheet included assessor comments such as used a range of tensesvocabulary sufficiently broad but these are too general 3.08Client Sample: Demonstrated Client was able to engage in conversation on familiar and some less familiar contexts Assessor comments need to be supported by reference to specific quotes of questions asked by the assessor and to the clients ability to maintain a conversation e.g. turn taking Where assessor noted asked questions to clarify discussed volunteer responsibilities a few examples of actual questions may have provided stronger evidence LWA 2014 11

Slide 12

PTA 3: TASK & CLIENT RESPONSE, LEARNING TASK TASK: Aligned Consider review of layout of Assessment Interview Form to provide more space for recording responses Suggest starting the Assessment Interview Form with questions relating to highly familiar, personal details to put low level clients at ease Prompts are too long if read as a block of text Assessor could have extended the conversation to demonstrate full extent of competence 1.01 Client Sample: Demonstrated Evidence supports reported indicator Further questioning, prompting may have allowed for higher competence to be demonstrated 1.02Client Sample: Demonstrated Evidence supports reported indicator Further questioning, prompting may have allowed for higher competence to be demonstrated Assessor could have made observations of strategies throughout the PTA LWA 2014 12

Slide 13

PTA 3: TASK & CLIENT RESPONSE, OC LWA 2014 13 1.07 Client Sample: Demonstrated Quotes on Assessment Interview Form support reported indicator Extensive supporting quotes are recorded on the scribed assessor sheet Ensure all assessor comments pertain to indicator performance features 1.08Client Sample: Demonstrated Assessor comments supported with referenced examples Evidence is strengthened with quoted examples of requests for clarification and repetition

Slide 14

PTA 3: TASK & CLIENT RESPONSE, READING TASKTASK: Aligned It would be useful to include a range of texts in a PTA kit relevant to the client groups Pre 1.03 Client Sample: Demonstrated Assessor comments address Writing and Numeracy performance features Ensure comments address the performance features of the indicator being claimed Pre 1.04 Client Sample: Demonstrated Some assessor comments give evidence for learning LWA 2014 14

Slide 15

PTA 4: TASK & CLIENT RESPONSE, LEARNING TASK TASK OC & LEARNING Assessment Interview Form: Aligned Suggest starting the Assessment Interview form with questions relating to highly familiar, personal details to put low level clients at ease Prompts are too long if read as a block of text Assessor could have extended the conversation to demonstrate full extent of competence A self-assessment form: Not aligned Not an appropriate task for a low level client Limited application against ACSF Performance features Assessor noted high level of support was provided. What form did the support take e.g. was the client expected to read the task but found it too difficult? Core LLN Skill Assessment Summary Sheet: Aligned Assessor has provided examples of client performance against the ACSF Focus Areas Pre 1.01 Client Sample: Demonstrated Assessor notes together with client performance support reported indicator Pre 1.02 Client Sample: Demonstrated Assessor notes together with client performance support reported indicator Assessor observed strategies throughout the PTA LWA 2014 15

Slide 16

PTA 4: CLIENT RESPONSE OC Pre 1.07 Client Sample: Demonstrated Based on evidence client sample aligned. Evidence on Assessor Notes sheet suggests more highly familiar questions may have produced evidence of 1.07 No clear delineation of comments addressing of individual indicators Pre 1.08 Client Sample: Not Demonstrated Client responses demonstrated understanding of several questions of varying complexity thereby indicating 1.08 was achieved Need to include a few examples of questions that need to be rephrased Assessor comments span both indicators. The.08 indicator is not clearly addressed LWA 2014 16

Slide 17

PTA 4: TASK & CLIENT RESPONSE READING TASK Both text types formatted. No prose text used Task: Emergency Procedures: Not aligned Task complexity spans several levels of reading In order to award Pre level 1, a task aligned to level 1 must be used Task: Calendar: Not aligned Task contains complex language Too many questions. Questions 5 11 do not address level 1 reading performance features Pre 1.03 Pre 1.04 Client Sample: Not Demonstrated Comment have difficulties if the text has unfamiliar vocabulary and complex structure indicates Emergency Procedures task is too complex The client may well have demonstrated competency at level 1 if appropriate texts were used given that she has answered questions 1 - 4 correctly LWA 2014 17

Slide 18

PTA 4: CLIENT RESPONSE WRITING TASK Task: aligned Two appropriate text types were used Parts of the form exceed level 1 complexity. To ensure consistency between assessors maybe modify the form so there is a clear distinction between the sections to show level 1 and level 2 Pre 1.05 Pre 1.06 Client Samples: Demonstrated Personal details on form copied as noted by assessor indicate the client is still pre- level 1.05 Although the client demonstrated some 1.06 features (as acknowledged in the assessor notes e.g. assessors comments relating to punctuation) pre-level 1 is appropriate e.g. the information in the form was copied LWA 2014 18

Slide 19

PTA 4: TASK & CLIENT RESPONSE NUMERACY TASK Task: Not Aligned Tasks are not real life. They consist of a series of decontextualized activities. Portions of the task are maths questions (questions 3-7) Some of the task activities are beyond level 1 (questions 15, 16, 17, 18, 200) 1.09 1.10 Pre 1.11 Client Sample: Demonstrated Some comments span indicator levels Assessor notes together with client performance supports reported indicators despite the unsuitable tasks LWA 2014 19

Slide 20

SESSION 3 Moderation Activities: LEARNING LWA 2014 20

Slide 21

SESSION 3: LEARNING TASKS 1-5 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? (If supplied) Would you adapt this task to use in your own context? How? LWA 2014 21

Slide 22

LEARNING TASK 1: TASK & CLIENT RESPONSE MY GOALS & HOW DO YOU LIKE TO LEARN 1.01 It is acknowledged that Distance learning presents particular challenges for assessment and documentation Tasks: Aligned Could be broadened to cover past experiences of successful learning, any barriers and planning and organising Task 2 provides evidence of preferred learning approaches but client may be confused by the graphic Task 2 could be redesigned to allow for demonstration of more focus areas Client Sample: Demonstrated Evidence would be strengthened if assessor noted any relevant details gained in conversation with the client Assessor notes map evidence to focus areas although the notes are very brief Assessor comment on strategies is a.02 performance feature Assessor could note clients ability to engage in learning activities with appropriate scaffolding as additional evidence for learner identity LWA 2014 22

Slide 23

LEARNING TASK 2: TASK & CLIENT RESPONSE BOOKLET & STRATEGIES FOR LEARNING 2.01Tasks: Aligned Covers a broad range of performance features Client Sample: Demonstrated Prompting and advice box ticked but no information provided Variation in scaffolding provided Assessor comments are succinct yet thorough with mapping of evidence to relevant performance features Assessor prompting with Qs 1 and 7 could have strengthened the evidence Updated SEE program task Cover Sheet should have been used LWA 2014 23

Slide 24

LEARNING TASK 3: TASK ONLY LEARNING IN THE WORKPLACE 2.02 Task: Aligned Depending on responses, the task allows for a range of levels to be demonstrated Good task to facilitate evidence from a work placement Task could be modified to suit a range of workplace contexts Some of the statements could be reworded as questions e.g. statement 1 to allow for a broader range of responses LWA 2014 24

Slide 25

LEARNING TASK 4: TASK & CLIENT RESPONSE RANGE OF ACTIVITIES 2.02Tasks: Aligned Task well constructed covering a broad range of performance features Client Sample: Demonstrated Responses may have been more detailed if the assessor had scribed the student responses Assessor Comments template is well structured Not all client responses show strong exit level 2 competence e.g. task 1, task B question e. However, there is sufficient evidence overall to support the outcome Some assessor prompting may have allowed for stronger responses e.g. completion of task 5 LWA 2014 25

Slide 26

LEARNING TASK 5: TASK ONLY MY WORKPLACE A REFLECTION 3.01 3.02 Task: Aligned Good task for gathering evidence from a work placement and task could be modified according to workplace context Could be used as part of a project about the work experience. This could broaden the possible demonstration a wider range of Learning focus Areas as well as demonstration of other Core Skills LWA 2014 26

Slide 27

SESSION 4 Moderation Activities: ORAL COMMUNICATION LWA 2014 27

Slide 28

SESSION 4: ACTIVITY, TASK 2 Q & A BOARD GAME FILM CLIP REMOVED You will see a group of students playing the Questions and Answers game. Pre-teaching includes practising question forms. After watching, discuss in your group: What OC outcomes can be assessed with this game? Are there any other core skills that can be assessed? How? How can the game be adapted to assess different ACSF levels? LWA 2014 28

Slide 29

SESSION 4: OC TASK 2 MODERATION: Q & A BOARD GAME TASK & CLIENT SAMPLE Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? Would you adapt this task to use in your own context? How? LWA 2014 29

Slide 30

OC TASK 2: TASK & CLIENT RESPONSE Q & A BOARD GAME 1.07Task: Aligned Creative task with potential to be used effectively across OC levels Instruction sheet provides guidance on participation (supported by examples on the white board. See video) Assessor sheet is very comprehensive and allows for reporting of strong evidence against the performance features Client Sample: Demonstrated Teacher notes provide strong demonstration of both the ability to ask questions and give short responses Assessor comments support reported indicator well LWA 2014 30

Slide 31

SESSION 4: ACTIVITY, TASK 8 ABC 7:30 REPORT FILM CLIP REMOVED This is a segment from the ABC 7.30 Report entitled Casualising the Workforce. Discuss: What OC listening level would you assess with this story? Could you also use it to assess speaking? How? Could it be used as a springboard to assess other core skills? How? Would you use this idea? How? LWA 2014 31

Slide 32

SESSION 4: OC TASK 8 MODERATION: 7:30 REPORT, CASUALISING THE WORKFORCE TASK & CLIENT SAMPLE Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? Would you adapt this task to use in your own context? How? LWA 2014 32

Slide 33

OC TASK 8: TASK & CLIENT RESPONSE CASUALISING THE WORKFORCE 4.08Task: Aligned Excellent task relevant to the needs of the learner cohort Context of task in classroom learning well explained Text demonstrates appropriate complexity for the indicator Could be strengthened in Focus Area of Audience and purpose. Consider some questions to elicit client understanding of the purpose and motives of different speakers Could be strengthened in Focus Area of Comprehension. Consider some question reformulation to allow client to clearly demonstrate ability to relate pieces of information in an oral text rather than treating them as separate pieces of information Evidence strengthened through the provision of a part transcript to give an indication of the complexity of the listening text. It is not necessary to include the entire transcript Client Sample: Demonstrated Demonstrates understanding of complex oral texts (multiple speakers) Understands abstract nouns, e.g. casualisation Understands vocabulary specific to the topic LWA 2014 33

Slide 34

SESSION 4: ORAL COMMUNICATION TASKS 1,3,4,5,6,7 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? (If supplied) Would you adapt this task to use in your own context? How? LWA 2014 34

Slide 35

OC TASK 1: TASK ONLY AT THE BUTCHER 1.07Task: Not Aligned Appropriate real life task Role play cards too heavily prompted to allow for demonstration of exit level competency Assessment Criteria sheet does not closely reflect indicator performance features LWA 2014 35

Slide 36

OC TASK 3: TASK & CLIENT RESPONSE G & S TRANSPORT TICKET 1.08Task: Aligned Authentic relevant task for learner cohort supporting community engagement Well structured and scaffolded task Student reflection on page 5 could potentially provide some learning evidence Client Sample: Demonstrated Assessor annotations together with quotes of clients speech strongly support indicator performance features Transcript of transaction provided Detail of classroom context provided on page 3 of task LWA 2014 36

Slide 37

OC TASK 4: TASK & CLIENT RESPONSE DENTAL APPOINTMENT & CASUAL CONVERSATION 1.07 Tasks: Aligned Clear instructions to learners Prompt cards provide appropriate level of support Client Samples: Demonstrated Strong evidence aligned to the relevant performance features both on the Task Cover Sheet and on the tasks Clear transcripts of conversations provided LWA 2014 37

Slide 38

OC TASK 5: TASK & CLIENT RESPONSE STUDENT RANG ME 1.07Authentic sample: Aligned Client Sample: Demonstrated Good example of capturing authentic evidence from the student Evidence supports the indicator claimed LWA 2014 38

Slide 39

OC TASK 6: TASK ONLY PHONE MESSAGE 2.08Task: Aligned Need to consider students with spiky profiles i.e. will writing ability impact on task outcomes? Task designed for use with multiple core skills (Listening, Writing, Reading) Listening text is appropriate context and complexity The message pad may be confusing as it does not readily relate to the information in the text e.g. while you were out could be changed to from LWA 2014 39

Slide 40

OC TASK 7: TASK ONLY PART TIME & CASUAL WORK 3.08Task: Aligned The task is highly relevant for students planning to enter the workforce Explanation of context in classroom learning would need to be explained on the Task Cover Sheet Question 8 could be reformulated to give students the opportunity to express their opinion on the benefit of this information to listeners Some level 4 complexity in the text but scaffolding of questions supports the 3.08 outcome LWA 2014 40

Slide 41

PLEASE FILL IN YOUR EVALUATION FORM Contact us: T: 03 9429 7551 F: 03 9429 7221 www.lwa.net.auwww.lwa.net.au SEE admin: see@lwa.net.ausee@lwa.net.au Office admin: louise@lwa.net.aulouise@lwa.net.au Verifiers: kath@lwa.net.aukath@lwa.net.au linda@lwa.net.aulinda@lwa.net.au mary@lwa.net.au fiona@lwa.net.au julie@lwa.net.au jane@lwa.net.au chris@lwa.net.au dina@lwa.net.au suzana@lwa.net.au LWA 2014 41