National Course design and assessment
SQA fieldwork visits — engagement and focus group discussions with centres delivering new National Qualifications
This edition: May 2016, version 1.0
© Scottish Qualifications Authority
1 Background 1
1.1 Acknowledgements 1
1.2 Rationale 1
1.3 Research methodology 1
1.4 Impact of curriculum structures and option choices 3
2 Executive summary of themes and issues emerging 3
2.1 Broad general education 4
2.2 Assessment 4
2.3 Personalisation and choice 5
2.4 Documentation 5
2.5 Developing the Young Workforce 6
2.6 Further support 6
3 Conclusions 7
4 Annexes 8
4.1 Questions for senior management team 8
4.2 Questions for teaching staff 10
4.3 Questions for candidates 12
4.4 Background questionnaire for teaching staff 13
4.5 Background questionnaire for candidates 15
4.6 Summary of senior management team focus group responses 17
4.7 Summary of teacher focus group responses 29
4.8 Summary of candidate focus group responses 35
4.9 Statistical data — teachers (background questionnaires) 43
4.10 Statistical data — candidates (background questionnaires) 44
4.11 Online survey analysis and commentary 46
1
1 Background
1.1 Acknowledgements Thanks to the Curriculum for Excellence (CfE) support team for their help in organising the
focus group visits, and in developing the focus group materials and questionnaires which the
teams used as the basis for their work in centres, and to the team leaders and scribes who
coordinated the focus group work.
Thanks should also go to colleagues in the Research, Policy, Statistics and Standards team
(RPSS) for helping to develop the questions used in the focus group work, and to the
Customer Service Managers team who helped in coordinating the school centre visits.
Above all, thanks to the senior managers in SQA centres for giving up their time to meet with
the teams and helping to organise the meetings with the focus groups. Also, many thanks to
the teachers and candidates for taking the time to meet with the teams, and complete the
detailed questionnaires.
1.2 Rationale This series of engagements with centres/colleges/local education authorities (LEAs) and
other partners is part of a suite of SQA activities designed to evaluate the first phases of the
new National Qualifications (NQs). Its purpose was to attempt to evaluate at first hand:
the impact that implementation of the new NQs has had on centres in the period 2012-16
what, if any, effects this implementation continues to have
Whilst not linked directly to the thematic review conducted by colleagues in RPSS, some of
the observations and data included here do have links to some of the data included in that
review, in terms of opinions gathered from National Qualifications Support Teams (NQSTs)
and an alignment of questions asked at interview.
1.3 Research methodology The Qualifications Development (CfE) team formed teams of three in cooperation with SQA’s
Customer Support Manager (CSM) team to visit centres in each local authority. A minimum
of one centre per authority was the target for this exercise, although the number was
extended in larger authorities. The centres were chosen to reflect the wide variety of contexts
with which centres operate, so the centres included in the research were a spectrum of rural
and urban; large and small by roll; denominational and non-denominational; reflective of free
meal entitlement provision. This resulted in visits being made to 40 centres (including the
results from two pilot schools) during November and December 2015.
It was also proposed that the teams have a structured conversation with a number of college
providers, parents, and local authority representatives to reflect their perspectives on the new
2
NQs. This process is still ongoing — meetings have not yet taken place with college staff and
other recent meetings are still being written up.
Each team visiting a centre consisted of a team leader (one of four in this role) who acted as
moderator, and a scribe from the Qualifications Development (CfE) team. This was to ensure
validity and reliability in the process of recording the focus group responses. In addition, the
CSM member acted in an introductory and liaison capacity and as an additional note-taker
where necessary. The questions for the three focus groups were developed by the
Qualifications Development (CfE) team in liaison with RPSS. The questions were sent to the
centres to allow the individuals and groups chosen to discuss and prepare any responses in
advance (and to compensate for limited time in some cases), with the proviso that the
moderators might prompt or probe deeper in conversations in individual cases where it was
felt appropriate.
The Qualifications Development (CfE) team asked for the groups to be formed in the
following way where possible, taking account of local circumstances:
8-10 pupils from S5-S6, ie those candidates who had a fuller experience of the new NQs
6-8 classroom teachers/principal teachers/faculty heads ie to reflect the balance of
teaching staff and the variety of departmental organisation across the country
as many of the senior management team as could be available on the day for the
discussions at school management level
The timing of the focus groups was agreed in advance with the centres to fit in with their own
timetabling structures. This meant the group’s discussion could vary from 40 minutes to one
hour. The pupil and teacher groups were also asked to complete a background questionnaire
(see Annexes 4 and 5) to provide information about certain aspects of their experience of the
new NQs, and any specific issues that had emerged for them over the implementation. This
was designed to add experiential depth to the data being gathered in the focus groups and
help to develop a picture across curriculum areas and subjects.
The design of the questions was intended to elicit and identify generic issues across centres
and areas of the country. Some of the questions posed were common to all groups to elicit
comparative responses. However, given the make-up of at least two of the groups
(candidates and teachers), it was accepted that some of the opinions expressed would be
derived, both in the questionnaire and focus groups, from specific subject-based
experiences. That may be reflected below, wherever appropriate, to illustrate any elaborated
responses. The design of the questions incorporated, in some cases, a specific set of
potential responses to help the scribes record the information, but these were not intended to
deflect the conversation from more elaborated responses where appropriate. Broad scope
was given for the occasionally complex recording of these.
The questions were piloted in two centres to help the Qualifications Development (CfE) team
to identify any initial problems, either with the questions or the timing of the focus groups, but
by and large the methodology chosen seemed to be appropriate to the task. In addition,
centres were invited to complete a set of online survey questions, in order to give greater
depth and detail to the data being gathered. The core of the online survey analysis and data
is included in Annexe 11.
3
Great care was taken to reassure centres that their contributions would be anonymised and
no references would be made which could identify where specific information had originated.
Accordingly, there are no school-specific references in this document.
1.4 Impact of curriculum structures and option choices It should be noted that the spread of centres included in the research presented a variety of
curriculum models and subject choices for candidates, and the candidate questionnaire
responses reflected this. Generally, centres stipulated that they were using a 3+3 model,
although it was apparent that, in practical terms, some centres still had candidates making
some form of option choice at the end of S2. In addition, it was clear that there was a
spectrum of arrangements at local authority level. Initially, it seems that most of the LEAs
had given centres a fairly free rein in developing curriculum models on a local basis.
However, it was also apparent in discussions that some LEAs were reviewing these
arrangements in order to design curriculum models and subject choices across the LEA.
Whether these changes were being made for strategic or cost reasons was not always clear
to the focus group teams.
In looking at the candidate questionnaires, it is clear that the subject choices being made for
the senior phase starting in S4 still reflect a high number of subjects being taken at that stage
— up to nine in some cases, reducing to five in S5 — with a variety of Higher and Advanced
Higher in S6. Some centres have subsequently reflected on this and have reduced, or are in
the process of reducing, the number of subjects in S4. Additionally, because centres are
‘compressing’ what they previously perceived as two years’ work into S4, there is a high
element of pressure to complete N4/N5 in two terms. Also, the five-Higher model in S5 might
suggest that there is still little movement away from the ‘two-term dash’ to Higher that was
common under the previous NQs. It could be inferred from this that the intention that the
senior phase should ‘open up’ the points where candidates sit qualifications has not yet been
achieved, and that structural, timetabling and staffing issues may be impacting on this.
2 Executive summary of themes and issues emerging This summary is intended to highlight a number of specific themes and issues emerging from
the focus groups/centres surveyed.
Evaluating and supporting the new National Qualifications is clearly in the interest of SQA.
However, a balance will need to be found between potentially conflicting priorities: such as
ensuring the continuing rigour of live qualifications each year; continuing to support centres
by events and documentation; and continuing to evaluate and gather evidence on the
implementation of the new qualifications.
This document also includes the structured questions that were used with centres in
Annexes 1-3, the background questionnaires sent in advance to teachers and candidates in
Annexes 4-5 and the detailed summaries of the various focus group responses in Annexes
4
6-8. Analysis of the statistical data gathered is included in Annexes 9-10, with online survey
analysis and data included as Annexe 11.
2.1 Broad general education It was found that the initial structure and content of the broad general education (BGE) in
S1-S3 had not, in a number of centres, prepared candidates well enough for the
demands of the senior phase, including qualifications. However it was stated by both staff
and candidates, that this pattern varied across subject areas.
The ‘step up’ from the end of S3 into S4 presented major challenges for many
candidates. Some candidates expressed the view that they felt they were ‘marking time’,
especially in S3, and that the pace of learning and assessment had picked up
dramatically in S4.
As a result of the above, many centres are now revisiting the structures of the BGE to
reflect the needs of candidates moving into the senior phase. In particular, discussion on
bringing a measure of content back into S3 was common, and similarly on a need to
improve skills development, which was an issue for some subjects receiving smaller time
allocations in S3.
In particular, many subject areas are looking to ‘map’ some aspects of content from NQ
courses onto the work being undertaken by candidates as part of the BGE. This appears
to be undertaken in order to help candidates cope with levels of content in some subject
areas. This appears to be a pattern across most subjects in order to cope, and help, with
the demands of Unit assessment in S4. (See further points in ‘Assessment’ below.)
However, there is concern in some centres that this change in practice might contradict
advice being given by Education Scotland about NQ work becoming part of the BGE, and
this is causing a degree of uncertainty for staff, and some concern if this rigid separation of
BGE and senior phase work is in the interests of candidates.
2.2 Assessment Many centres commented on the lack of clarity in the relationship(s) between Unit
assessment and other Course assessment, including the question paper (QP). This is
particularly true where they feel Unit content or assessment is not reflected or linked to
work for question papers. This may also be due to misunderstanding the interrelationship
but also the separation of parts of the Course.
Candidates and centres felt that Unit assessments were not adequate preparation for the
question paper — where one existed — as candidates believe they are benchmarked ‘too
low’ (C level or minimum competence).
Most centres and subjects are still using Unit-by-Unit assessment, although some are
branching out to Combined and Portfolio (Business and PE were mentioned). However,
some subjects who had attempted Combined and Portfolio approaches initially had found
recording of results highly complex in practice and had reverted to the Unit-by-Unit
approach.
Many centres and candidates commented that Unit assessment is burdensome, took up
a lot of time and, particularly in the early stages of implementation, the pressure of this
had been very stressful. Many centres and candidates commented that the need to pass
5
all Assessment Standards in a Unit was unrealistic, putting great pressure on teachers
and candidates alike, and leading to considerable re-assessment for candidates.
Many teachers expressed the opinion that assessment was ‘verification-driven’ —
meaning that the expectation of being verified for a level meant they ‘had to get it right’.
Many candidates expressed the feeling that their teachers seemed ‘under pressure’ over
the level of assessment, although this had improved over time.
There was a strong feeling across all groups that the lack of a question paper at N4 gave
it a lack of credibility, particularly in the cases of bi-level teaching where the value placed
on the Added Value (AV) Unit for N4 was viewed differently by N4 and N5 candidates
and by teachers.
Paradoxically, the need to have candidates achieve all Unit Assessment Standards in the N4
AV Unit, in order to gain a National 4 Course, may in fact have created an imbalance of
expectation in contrast to those candidates at N5, where the Course can be successfully
achieved without candidates needing to gain 100% in all Course Components.
2.3 Personalisation and choice Most candidates did not understand what was meant by personalisation and choice
(P&C) and in many cases the purpose of the question had to be elaborated before the
focus group discussions could continue.
Many candidates expressed the view that P&C had been experienced more in the BGE
than in the senior phase, mainly due to the level of content in some subjects
(Mathematics and Sciences being mentioned in particular).
Teachers felt that they could only offer a limited amount of P&C in the qualifications.
Open choice exposed the problems of resourcing/monitoring and expertise in some areas
when it came to making assessment judgements.
Candidates and teachers felt the scope for P&C in the assignment was limited, both in
terms of time and their ability to choose freely a topic for study. Some candidates referred
to a ‘managed choice’ where a list of options was made available to them. A general
opinion though was that P&C varied widely across subject areas.
Subjects who had initially offered an open choice in preparation of assignments, etc, had
now restricted P&C to keep things ‘manageable’ and utilise their resources.
2.4 Documentation There was a general feeling among staff that there was excessive documentation and
that repetition (particularly of SQA policy positions or template language, eg Equality) in
documentation was unnecessary and much of it superfluous.
Staff expressed a strong feeling that a major rationalisation of the documentation was
needed. The most useful documents were felt to be Course and Unit support notes and
Unit assessment support materials (UASPs), even though these are not mandatory.
Some comments about the accuracy of individual specimen question papers and marking
instructions were made. The format and structure of the specimen question papers and
exemplar question papers not being reflected in the ‘live’ papers was perceived to be an
issue in some subjects.
Some staff commented that they were grateful to see that SQA was taking notice of
comments being made, and this was impacting on changes they were seeing.
6
There were many comments about the timing of change to documents. Most people felt
that the notification of changes to documents was coming too late in the term, causing
some centres to have to re-assess candidates when changes were realised. The level of
change to documents was also heavily commented on, as it was difficult and time-
consuming to keep track of changes.
2.5 Developing the Young Workforce Most centres thought that they had appropriate provision in place to deliver a range of
vocational qualifications.
It was not clear that these were always specifically aimed at the purposes and timescales
of Developing the Young Workforce (DYW).
Some centres expressed the view that the recent changes to college structures had
impacted to some degree on their school-college partnerships, which for most are the
main vehicle for delivery of vocational qualifications. In particular, local authority
representatives noted the impact of colleges requiring minimum numbers for courses to
run, which centres could not always match.
The view was expressed in some centres that pitching the new Foundation
Apprenticeships at SCQF level 6 may be having a detrimental impact on access to
college places for less able pupils, who formerly might have been part of the school-
college cohort.
Some local authorities were reverting to consortia-type arrangements to help mitigate the
issues arising from college re-organisation.
Centre preparation to deliver DYW varied widely across the country, as did the level of
school/college links.
2.6 Further support Staff and senior management teams expressed a general feeling that the additional
support implemented by SQA was valuable and should continue. There was particular
praise for Understanding Standards events and their value as Continuing Professional
Development; there was a strong feeling that they should be continued.
Access to Understanding Standards events varied across the country, with the more rural
and remote parts of the country finding the most difficulty in attending as this often meant
overnight stays. In addition, there are issues of cost and cover for centres where multiple
events are held on the same day. It was also noted that staff in some centres had
difficulty accessing the Understanding Standards events due to limits on numbers.
Verification and Course reports were found to be most useful, although depth and quality
varied widely across subjects.
Visiting verification, where it is used, was thought to be the most helpful. The chance to
have a detailed conversation with verifiers was welcomed to help understand how
decisions had been reached. Telephone conversations with verifiers were not always
viewed as helpful.
Some centres had found support from Subject Implementation Managers (SIMs) to be
useful, but there was a wide spectrum of knowledge about SIMs and how their support
could be accessed.
If centres had nominees, they were found to be very useful in helping staff to understand
standards.
The use of nominees varies widely across authorities, so support for centres is variable.
7
3 Conclusions More detail from the focus groups and advance questionnaires is available in the
accompanying annexes. The high-level messages in the executive summary have been
extracted to inform short- to medium-term discussions on those issues only. Other issues for
action may emerge in conjunction with other data gathered from the thematic review and
evidence from the online survey work.
However, given that SQA is an evidence-based organisation, there is scope for work of this
type to be extended longitudinally.
8
4 Annexes
National Courses Design and Assessment
SQA fieldwork visit
4.1 Questions for senior management team Focus group questions — for discussion The SQA team may prompt the conversation with additional information where necessary. For example where the SMT feel there are differences between subjects or across the school. 1 Considering your experiences across the whole school: a) How well do you feel the pace and content of learning in S1-3 is preparing candidates for
S4 and beyond — in terms of both knowledge and skills and attitude to learning? (Optional question) SQA believes there is a progressive demand between levels of qualification in a subject. Could we explore this with you as an SMT monitoring the delivery of the Courses? 2 SQA exemplified three different approaches to Unit assessment – Unit-by-Unit,
Combined and Portfolio. a) How are these different approaches being used within the school? b) How confident do you think teachers are now in assessing Units? (Is that changing over
time?)
3 The new National Qualifications (NQs) include Unit assessment as well as Course
assessment (N4 Added Value Unit, question papers and coursework). a) How are these various assessments managed across the school year and across
different subjects? b) How confident do you think teachers are in marking coursework and in estimating
grades?
4 Schools are adopting different models for their delivery of the new NQs. a) Could you comment on your own delivery model and any changes you might be
considering (eg number of presentations for each year group, time for teaching, bypassing work for NQs in S3, presenting for both N5 and N4 Added Value Unit, etc).
9
5 The ‘Developing the Young Workforce’ report and the CfE Implementation Plan suggest that schools introduce more vocational qualifications and establish links with colleges and employers.
a) Can you talk to us about your plans? b) How can SQA support you in this change process? 6 SQA support for the implementation of the new NQs includes a range of mandatory
and support documents, and additional support such as the wider role of nominees, verification reports, Understanding Standards events and Course reports.
a) When talking to departments, what is your perception of how useful and supportive these
have been on influencing departmental practice and leading to better outcomes for pupils?
7 Is there anything else you’d like to tell us about your experience of the new NQs or
any other SQA qualifications?
10
4.2 Questions for teaching staff Focus group questions — for discussion
The SQA team may prompt the conversation with additional information where necessary. For example where the staff feel there are differences between subjects.
1. Thinking about your own subjects: a) How well do you feel the pace and content of learning in S1-3 is preparing candidates
for S4 and beyond — in terms of both knowledge and skills and attitude to learning? b) How smooth is the progression in demand between the different levels of a subject?
2. SQA exemplified three different approaches to Unit assessment — Combined, Unit-by-Unit and Portfolio.
a) What do you feel are the advantages and disadvantages of each approach for your
subjects?
3. The new National Qualifications (NQs) include Unit assessment as well as Course assessment (N4 Added Value Unit (AVU), question papers and coursework).
a) What is your experience of balancing the work and assessment of the Units with
preparation for and undertaking the Course assessment?
4. Schools are adopting different models for their delivery of the new NQs.
a) Is any work on NQs being done in S3 and, if so, what impact has this had on your candidates?
b) Have any of your pupils undertaken the N4 AVU in the same year as the N5 assessment for that subject and, if so, are you continuing with that approach?
5. The documents for the new NQs were published in sequence for each subject and level, from Course and Unit specifications to Unit and Course assessment support.
a) Which documents have been of most use to you? b) Now that we are much further into the implementation cycle, would you welcome some
rationalisation of the documents?
11
6. SQA support for implementation of the new NQs includes the wider role of nominees, verification reports, Understanding Standards events and Course reports:
a) How useful have these been to you?
7. The new NQs seek to encourage a different way of learning and teaching in the senior phase (as in the broad general education):
a) What is the impact of personalisation and choice, and of opportunities for independent learning and self-reflection?
8. Is there anything else you’d like to tell us about your experience of the new NQs?
12
4.3 Questions for candidates Focus group questions — for discussion
The SQA team may prompt the conversation with additional information where necessary.
For example where they feel there are differences between subjects.
1. You are all in the senior phase — S4 and above:
a) Thinking about what you learned in terms of skills and knowledge in S1-S3, how well
did that prepare you for your work in the senior phase?
b) Did the speed of learning in S1-3 prepare you for the work in the senior phase?
2. Thinking about the qualifications you are doing this year:
a) How is the step up from what you were doing last year?
3. Now think about the work you did last year:
a) What was your experience of assessment during last year?
4. Unit assessments are designed to help you prepare for Course assessment:
a) How do you feel the Unit assessments helped you to prepare for Course assessment
(that is for question paper and/or coursework)?
b) If you were re-assessed for any Unit or part of a Unit, how did that happen?
5. Now think about the subjects you are doing this year, or those you did last year:
a) How much personalisation and choice do you feel there is in how you’re learning and
what effect does that have?
6. Is there anything else you’d like to say about your experience of the new
National Qualifications?
13
4.4 Background questionnaire for teaching staff Background information — to be completed in advance of the focus group
Which subject(s) do you teach and to what level (eg Biology to AH and Chemistry to N5)?
Do you teach any vocational qualifications? (select as appropriate) Yes ☐ No ☐
If ‘Yes’, which subject(s) do you teach?
What is your position in school (eg Faculty Head of Science, PT English, Teacher of
Computing)?
Are you an SQA appointee? (select as appropriate) Yes ☐ No ☐
If ‘Yes’, please give details
Are you an SQA nominee for verification? (select as appropriate) Yes ☐ No ☐
If ‘Yes’, please give details
Have you attended any SQA events? (select as appropriate)
☐ Implementation Events for N3-N5, Higher, Advanced Higher
☐ Understanding Standards Events for N5, Higher, Advanced Higher
Please list any other SQA events you have attended in the last three years:
Have you developed prior verified material in your department/curriculum area? (select as
appropriate)
Yes ☐ No ☐
14
Do you use the N4 Added Value Unit for your N5 candidates, and if so, how?
Which aspects of SQA’s senior phase assessments and qualifications have been the most
challenging and the most positive in your department/curriculum areas?
In SQA-produced Unit assessment support materials, three approaches to assessment
have been exemplified — Combined, Unit-by-Unit and Portfolio.
Combined assessment is the assessment of the Outcomes and Assessment Standards from more than one Unit within a Course or across Courses.
A Unit-by-Unit approach assesses each Unit separately and there is normally one assessment per Unit.
A Portfolio approach focuses on the collection of evidence across individual Units and the Course.
In all three approaches, assessment evidence may be collected as part of the day-to-
day learning and teaching process; this is referred to as naturally occurring evidence.
What has been the most common approach in your subject(s) and why?
What are the challenges in using the other approaches and why?
15
4.5 Background questionnaire for candidates Background information — to be completed in advance of the focus group
Current year (select as appropriate) S5 ☐ S6 ☐
What subjects and levels did you take in each year (eg N4 Mathematics, N5 Biology)?
Please include any other qualifications (eg Skills for Work Courses, other Awards).
S4 S5 S6
Subject Level Subject Level Subject Level
Others:
On a scale of 1-5, rate how hard you had to work in S4? (1 for not very hard, 5 for where you
had time for little else, working at evenings and weekends.) Please select one:
1 ☐ 2 ☐ 3 ☐ 4 ☐ 5 ☐
Then in S5? Less hard ☐ Just the same ☐ Harder ☐
Then in S6? Less hard ☐ Just the same ☐ Harder ☐
We will explore your collective answers to this question during the focus group discussion.
16
Have you used the SQA website to get information about your subjects? If so, how easy was
it to find what you wanted on a scale of 1-5 (1 being easy, 5 being very difficult)? Please
select one:
1 ☐ 2 ☐ 3 ☐ 4 ☐ 5 ☐
Have not used ☐
If you used the site, what information or support were you looking for?
Did you use any other websites? If so, which website(s)?
17
4.6 Summary of senior management team focus group responses
1 Considering your experiences across the whole school:
a) How well do you feel the pace and content of learning in S1-3 is preparing candidates for S4 and beyond — in terms of both knowledge and skills and attitude to learning?
Since the introduction of the new National Qualifications (NQs) almost all schools have had
to revisit and revise, or are in the process of revisiting, revising and refining, to a greater or
lesser extent, what they are teaching in S1-S3 in the broad general education (BGE). In the
majority of schools, SMTs indicated the pace and content of learning in S1 to S3 had not fully
prepared pupils for the new NQs and the lack of pace and content had been causing some
difficulties in the senior phase. There was recognition that S1-S3 should be laying the
foundations for a gradual increase in learning to avoid a large step-up to NQs. There were a
number of comments about ‘a disconnect’ and an ‘abrupt change’ between BGE and
qualifications and that phases are seen as distinct rather than fluid. In a significant number of
schools, BGE courses are now being made more skills-based and focused to assist in
creating a smoother transition to the senior phase.
During the first year of implementation of National 4 and 5 qualifications it also became clear
in a number of schools that the breadth, content and pace of Courses in S3 did not provide
enough depth for a smooth transition to S4. ‘The skills level in some subjects hadn’t prepared
them.’ In addition, some SMTs indicated that in subjects where there is a high level of
content, the transition from BGE to NQs can be more difficult and specifically referred to
Mathematics and Sciences. There was particular concern in these curriculum areas about
Courses being content-driven, that there may be a lack of time for pupils to develop skills as
a result, and the quantity of content may be putting staff and pupils under pressure in the
senior phase.
In particular, SMTs are working with staff to enrich the BGE with skills and knowledge, and
incorporate higher-order skills in the early years of school, that will help to prepare pupils for
the senior phase. ‘The qualifications have had a real impact on skills development and
therefore the BGE needs to have more emphasis on skills.’ One school has introduced
integrated course in S3 to concentrate on generic skills development. Schools are working
on skills progression, including working with associated primary schools, some are
introducing skills frameworks across curriculum areas and there was general feedback about
teaching being more pupil-focused, less didactic and that there is now more group work than
in the past. ‘It is percolating down, to ensure those skills are in place.’
A number of pupils are now being given 4th-level content from S3, mapped from the CfE
Experiences and Outcomes to the new qualifications, with specialism in some subjects being
introduced in S3 in some schools. There was some concern about the lack of preparation in
S3 for the SQA assessments pupils would encounter in S4 and beyond. ‘…what it doesn’t
prepare them for is the amount of assessment in S4.’
18
Teachers are beginning to use the language and vocabulary used in (CfE) assessments
earlier so that ‘pupils are conversant with the language’.
SQA believes there is a progressive demand between levels of qualification in a subject. Could we explore this with you as an SMT monitoring the delivery of the Courses?
A number of SMTs commented that the jump from National 5 to Higher is difficult, particularly
for National 5 pupils who had achieved a ‘C’ Grade pass. However, there was general
agreement that there has always been a big jump that many new NQs better prepare pupils
and there is better articulation and progressive demand between levels of qualifications.
SMTs were generally positive about progression but this is ‘overshadowed by assessment.’ It
should be noted that, as this was an optional question during fieldwork visits, this area was
only explored with approximately half the schools visited. It would perhaps be better to rely
more heavily on teacher responses to this particular question.
Having said that, there was general concern in schools about articulation of content and
progression from National 4 to National 5. This was highlighted particularly where pupils are
taught in bi-level classes, or where pupils who were expected to achieve National 5 had to
‘fall-back’ to National 4 late in the academic year. It appears to be difficult to move pupils
back between the different levels of qualifications in some subjects, eg from N5 to N4.
Sciences and Mathematics were said to be particularly problematic due to the amount of
content. English was also mentioned.
Decisions about presentations for specific levels are being left as late as possible. This can
lead to eventual N4 pupils experiencing assessment at both N5 and N4, and feeling over-
stressed and under-valued.
A number of schools commented they are exploring ‘next steps’ for National 5 ‘C’ Grade
pupils who will not be able to make the step up to Higher qualifications, including giving
pupils more curricular breadth and adding a Wider Achievement column in the school option
form.
2 SQA exemplified three different approaches to Unit assessment — Unit-by-Unit,
Combined and Portfolio.
a) How are these different approaches being used within the school?
In the vast majority of schools, class teachers initially used, and many are continuing to
choose, the Unit-by-Unit approach as this most closely correlates to the former National
Assessment Bank materials (NABs). Some staff were originally trying to replace NABs with
Unit-by-Unit assessment support materials. SMTs are leaving decisions about which
approach to follow to their staff.
As time has progressed and ‘things settle down’, practice is evolving and there is evidence
19
that teachers are now exploring other Unit assessment approaches, eg teachers ‘chunking’
assessment where they think it lends itself to the Portfolio approach (sic). Some are
gathering naturally occurring evidence. Generally, staff are using the approach they think is
most appropriate for their subject and Course level. Whilst there is increasing teachers’
confidence, it is not always possible to offer an approach that best suits individual pupils, as
this is hard to manage.
There was a significant amount of criticism about the changes made to Unit assessment
support materials during the academic year, once Courses had started. Changes mid-year
have caused confusion, reduced staff confidence and resulted in increased stress and
anxiety for teachers. This was highlighted as being an issue from a staff welfare perspective.
Staff are also experimenting with different Unit assessment approaches in an attempt to
reduce the recording, tracking and administration of Unit assessments. Tracking and
recording of Assessment Standards in Unit assessments is ‘an administrative nightmare’.
SMTs recognised that teachers are finding it very challenging to identify pieces of evidence
and to track and record pupils’ achievement of Assessment Standards. The number of
Assessment Standards is an issue in a range of subjects.
SMTs are aware that teachers and pupils are finding Unit assessment very burdensome and
there is a huge concern about the volume of Unit assessment. There was recognition that,
understandably, when qualifications are new, this can also lead to over-assessment. As all
the individual Assessment Standards have to be achieved in a Unit assessment, a significant
amount of time and effort can be spent on re-assessing a pupil on the one Assessment
Standard they failed. The assessment of Units is impacting on teaching time in many
schools. Teachers feel they are overloaded with assessment and managing re-assessment.
Unit assessment is seen by many as excessive and ‘interruptive’ of normal learning and
teaching. ‘Assessment is never-ending.’ ‘There is never an escape.’
A significant number of SMTs indicated that centre-devised A/B Unit assessment tests are
being used in addition to minimum competence Unit assessment, to help candidates prepare
better for the Course assessment. They reported that a majority of teachers are finding Unit
assessment is not enhancing learning or teaching, there is ‘no added-value to pupils’, they
‘were just something to get through’ and were not a good preparation for Course
assessment. The view was that Unit assessment in general is not articulating with
coursework and question papers.
Views were also expressed about the validity and use of open-book Unit assessments which
can be passed easily with pupils subsequently not performing well in Course assessment.
b) How confident do you think teachers are now in assessing Units? (Is that changing over
time?)
On the whole, SMT felt that teacher confidence is improving over time. Nominee advice,
verification feedback, Understanding Standards materials and events, and visits to schools
by Subject Implementation Managers (SIMs) have re-assured teachers, improved their
20
confidence levels and have, in some cases, led to changes in approach.
However, there is concern that documents are still changing, and considerably so in some
subjects. Courses start, based on published documents, and then at Understanding
Standards meetings some teachers discover the goalposts have moved or information can
often be contradictory to that which is published. Assessments ‘pulled’ from the SQA
website, are ‘now not deemed to be best practice’ and this reduces staff confidence.
Generally there were seen to be too many changes to documents and communication
messages need to be simplified. Having said that, some SMTs said that modifications to
documents had been welcomed and ‘it shows SQA was listening’ to feedback.
3 The new NQs include Unit assessment as well as Course assessment (N4 Added
Value Unit, question papers and coursework):
a) How are these various assessments managed across the school year and across
different subjects?
Almost all SMTs indicated they discuss assessment at team meetings. All SMTs agreed
there is too much assessment which is particularly due to some subjects having too many
Outcomes and Assessment Standards. The benefits of Unit assessment were seen to be
completely ‘outweighed by the amount of staff and pupil energy required.’ The amount of
assessment is leading to decreased learning and teaching time.
SMTs reported some pupils ‘collapsing under the burden’, ‘crumbling’ under the pressure
and experiencing ongoing pressure. Concerns were raised via guidance departments and
pastoral teams about pupils’ increasing levels of stress and pupils seeking support for this.
Parents have also contacted some schools expressing concern about the mental health of
pupils as a result of pressures.
Schools are attempting to manage, with varying degrees of success, the spread of
assessment, with many schools having implemented an assessment calendar. On some
occasions, assessment appears to have fallen into a natural spread. Where schools have
implemented an assessment calendar they are still trying to give flexibility to assess when
pupils are ready. These calendars are shared with pupils and parents. However, it was
reported by some that it is becoming increasingly difficult to provide support for learners, eg
scribes.
Many schools have not introduced, or have been unable to introduce, an assessment
calendar and there is still an ‘overwhelming’ assessment workload in February and March
each year. Obstacles to implementing an assessment calendar include finding agreement on
timing of assessment across subjects; subject competition for a calendar place; demands of
assessment for different levels of a subject; timetabling issues and different demands of
Course Components within a subject. In addition, SMTs indicated it is difficult to spread the
assessments out as pupils don’t have all the skills and knowledge to prepare for assignments
any earlier in the year.
A significant number of schools are using the National 4 Added Value Unit with S3 pupils and
21
some are still putting whole cohorts of National 5 pupils through the National 4 Added-Value
Unit too, as a ‘safety net’.
A lot of pressure is caused trying to fit three Unit assessments and coursework into nine
months in S4. Some schools are completing some Unit assessment in S3 to ease the
pressure.
b) How confident do you think teachers are in marking coursework and in estimating
grades?
There was a general feeling among SMTs that, over time, teachers are becoming more
confident in marking coursework. This has been helped by the fact that many staff are SQA
appointees. However, there were a number of comments around the lack of exemplification
materials, making teachers unsure about the criteria being applied when marking.
Schools have been finding internal moderation and concordance, in school and across some
individual LEAs, helpful. It was reported that staff are being encouraged by SMTs and
department heads to take up SQA marking duties as this would be beneficial to departments
within schools, whilst others have volunteered to mark to get information ‘from the horse’s
mouth’ and are able to cascade this information to colleagues. However, there are significant
resource implications in allowing staff release from school for marking duties as staff cover is
a persistent issue.
Staff appear to be becoming more comfortable with estimating grades. However, in some
subjects the disparity between the Unit and coursework content, and the lack of
exemplification, makes estimating more challenging. Estimated grades can now be
problematic as Unit assessment is not indicative enough for departments to make adequate
predictions in their estimates for whole Courses. Schools are using prelim grades to help
establish predicted grades.
4 Verification in the first year had three rounds and we have changed this to a round
of Understanding Standards plus two rounds. Do you feel that teachers are more
confident in their pass/fail judgements now in terms of Unit assessments and in
their estimates for the whole Course?
SMTs generally indicated they are happy with the current selection model for verification and
the demand on staff and pupils. Verification has generally been seen as helpful and
supportive. Two rounds of verification is better and ‘the whole process is settling down’.
However, the paperwork associated with verification is ‘onerous’ and some staff still ‘fear
verification’.
The quality and conduct of verification was highlighted by some as being an issue, with a few
verifiers being rather ‘unpleasant’, ‘disappointing’ and unsure of the standard. However, on
the whole, visiting verification was generally seen as being a positive experience and ‘they
have been very professional and have highlighted areas of good practice … really impressed
22
with SQA staff’.
Some schools found the feedback and messages mixed and the quality and content of
verification reports varied greatly from subject to subject. Some reports were not seen as
being very helpful as they did not provide sufficient details for the grounds on which the
decision was made. ‘Verification reports are all very different and there is no consistency.
Some were bog-standard and others were more personalised to the school.’
The timing of the feedback from March/April verification, which was provided mid-end June,
was considered far too late as it was so close to the end of the academic year.
It was generally agreed that having SQA nominees and markers in the staff was helpful; a lot
of schools were using these members of staff to support verification in school, and are giving
them release.
5 Schools are adopting different models for their delivery of CfE qualifications:
Could you comment on your own delivery model and any changes you might be considering?
(eg number of presentations for each year group, time for teaching, bypassing work for NQs
in S3, presenting for both N5 and N4 Added Value Unit, etc.)
The majority of schools have reflected, or are now reflecting, on earlier structural decisions
they made in terms of the school’s delivery model, and many are introducing changes as a
result. A few schools are bypassing qualifications at N5 with more able pupils; however, few
have any plans to introduce this at present.
A number of SMTs indicated that some National Qualifications work and assessment is
taking place in S3, ‘a lot of content is being taught in S3’ and that many of their staff would
prefer a 2+2+2 model as opposed to a 3+3 model. Higher has always been a two-term dash
and there was concern that this shouldn’t happen in S4 with National 4 and 5.
In the 40 schools visited, the number of NQs timetabled and studied for in S4 ranged from
five subjects to nine subjects and a minimum of five subjects in S5 in all schools. In each
senior phase year, pupils may also choose non-NQ courses. There was a mixed picture in
terms of whether LEA-wide decisions dictated the delivery model or whether this was
determined at a local level. Some authorities are exploring with their schools a whole-cohort
6/6/6 column model so that candidates can choose up to six subjects in each of these years.
A number of schools have introduced time within their timetable which they feel can enhance
personalisation and choice, coaching, mentoring and one-to-one contact between teachers
and pupils. A variety of timetabling models have been introduced — eg a 33-period week, an
asymmetric week — with the aim of providing a better balance and more contact time for
candidates.
A number of schools are operating, or plan to operate, a single cohort model or indicated
they had or are exploring a single cohort for the senior phase. However, there are timetabling
issues which they feel may lead to a reduction in personalisation and choice. A number of
23
schools highlighted that, over the next year or two, they will be offering some Highers over
two years, presenting pupils for Higher in S6.
Broad general education is being provided in all schools, with some still retaining the option
choice at the end of S2. Some schools introduce personalisation and choice in S3 when
pupils reduce the number of subjects, before reducing the number again for NQs in S4.
Despite the resulting assessment burden, many schools are attaching National 4 Added
Value Unit (N4 AVU) assessments to some N5 Courses and, in a few schools, to all N5
Courses. Some LEAs have taken the central decision to enter or not enter all S4 pupils for
both the N4 AVU and the N5 Course. N4 AVU is being used as both preparation and
progression with N5 pupils, and also as a safety net should it be required for ‘fall-back’ later
in the academic year. Many schools are using this with pupils towards the end of S3 and
‘bank’ materials. ‘Some departments are using National 4 Added Value as a step-up and
progression to the N5 assessments.’ One SMT advised that the local education authority is
looking at whether there should be N3 certification in S3. One school advised that they are
presenting pupils for a Languages for Life and Work Award in S3.
In some centres it was felt that the pressure to achieve N5 in S4 has pressured the centres
into a ‘seven-month dash’ and some less able pupils have become disengaged. There was
also some concern expressed about more able pupils becoming disengaged as they are
either ‘marking time’ and ‘treading water’ in S3, or they are wanting to be stretched in terms
of pace and challenge, but are ‘held in a box’ during this time. The comments that ‘It is very
frantic with high stakes tariff qualifications to do in one year’ or ‘It is a ‘constant battle trying
to fit two-year Courses into one-year slots’ perhaps are indicative of the pressures that some
structural models are causing for staff and pupils in some centres.
6 The Developing the Young Workforce report and the CfE Implementation Plan
suggest that schools introduce more vocational qualifications and establish links
with colleges and employers.
a) Can you talk to us about your plans?
The majority of SMTs have a range of vocational qualifications running within their schools
and have established links with colleges and employers. In general, links have been
established locally with colleges and universities to offer a wide variety for Courses such as
Fashion and Retail; Hospitality; HNCs in Computing and Dance; Skills for Work (SfW)
Aquaculture, Childcare, Construction, Engineering, Hairdressing, Maritime, Rural Skills,
Uniformed Services; Food and Drink Award. There were many examples of pupils studying
Advanced Higher qualifications in colleges.
Within school there is wide use of SQA qualifications such as awards in Employability,
Leadership, Personal Achievement, Personal Development and Volunteering. SMTs
indicated they are also using provision from other awarding bodies and organisations such
as the John Muir Trust, Duke of Edinburgh Awards, Saltire Awards, Skillforce, ASDAN and
the Open University (YASS modules).
24
However, some schools said that as yet they had little strategy or structure in place for DYW
and do not have a good range of vocational qualifications and established links with colleges
and employers. Distance for college campuses, difficulties with travel and transport costs as
well as having to make timetable adjustments for pupils were cited as barriers to
implementing DYW more fully.
Many schools have established links with employers with some pupils embarking on
Foundation and Modern Apprenticeships. There was some comment from SMTs that, whilst
Foundation Apprenticeships are felt to be a good idea, they are pitched at too high a level at
SCQF Level 6.
Some schools have good links with individuals and organisations such as the youth worker in
Community Learning and Development (CLD), Skills Development Scotland (SDS).
Some schools with good links indicated the negative impact college restructuring has had on
their DYW courses and programme in terms of capacity. There was also a suggestion that it
is ‘difficult to ensure pathways stay open year after year’. Minimum or threshold pupil
numbers can be a barrier to access further education (FE) courses as they are not deemed
viable.
A number of LEAs have provided funding which has allowed SMTs to appoint DYW or similar
PT posts. In addition, a number of LEAs coordinate vocational provision for all schools within
the authority and produce a timetabled senior phase partnership.
SMTs in some schools said ‘self-found’ work experience is completed by pupils.
b) How can SQA support you in this change process?
The following suggestions were made by individual SMTs in the schools visited:
1. SQA could provide support by putting in place a system for partners to be approved as
delivering centres (eg employers), which would allow them to undertake some of the
assessment.
2. SQA shouldn’t have done away with SVS which was a ready-made course, perfect for
life-skills and confidence-building, and not replaced it. Alternatives aren’t graded.
3. The centre currently does several awards but it has some budget-related constraints to
do Wider Achievement Awards, so it would be greatly appreciated if SQA could alleviate
some of the charges so that they can provide better opportunities to candidates.
4. We sent one boy to SQA for work experience and they put him through an online health
and safety assessment and this appeared on his certificate. We would like to find out if
there are more courses available online.’
5. SQA should keep information coming. Some courses look scary so get those who have
done them to share best practice.
6. Would like awards that ‘generalists’ could teach.
7. SQA should try to demonstrate parity between vocational and academic qualifications.
Parents don’t ‘get’ SCQF. SQA could come to parents’ nights to explain. Universities
could help by valuing these awards.
25
8. SQA could provide more support materials for awards to help schools get the work
started and the courses up and running.
9. We need SCQF Level 5 and 6 qualifications in employability (without enterprise) and
personal coaching and mentoring.
10. Accreditation of employability awards by SQA would be very useful for any youngsters
who go out to do work placements and vocational qualifications.
11. It would be very helpful if SQA could provide information on how to accredit some
bespoke/specific work done by candidates.
7 SQA support for implementation of the new NQs includes a range of mandatory
and support documents, and additional support such as the wider role of
nominees, verification reports, Understanding Standards events and Course
reports:
When talking to departments, what is your perception of how useful and supportive
these have been on influencing departmental practice and leading to better outcomes for
pupils?
The vast majority of SMTs in schools agreed that these wider supports have impacted well
on the school, particularly Understanding Standards events. There were a number of
comments about staff being ‘overwhelmed’ by the amount of wider support that is available.
Almost unanimously, the strong message from SMTs was there are far too many documents,
finding the information on the SQA website is difficult and confusing, and late changes and
information from SQA ‘torments the life out of people’. ‘There is a document which tells you
what all the changes are — it is difficult to find.’ Constant change to documents is very
unsettling for staff and can cause significant issues. ‘Colleagues are overwhelmed by
updates coming out of SQA on a weekly basis’, the ‘constant shifting sand’ together with ‘the
difficulty is keeping track of what’s changed’. The continual changes to documents, and far
too many changes, leaves staff anxious about missing changes: ‘continuing change is a real
stress to staff and kids’ and it is ‘very difficult to get your head round it when things keep
changing’. Particular subjects highlighted were Art and Design, Computing Science, Design
and Manufacture, English and Modern Studies. People felt that the goalposts seemed to
have moved quite markedly during the year in some coursework and this also resulted in
teachers losing confidence. There was also comment about errors in documents.
The course documents are ‘messy’ and it is hard to keep track of all the changes. Staff
constantly have to go back onto the SQA secure website and download Unit assessments
because of changes. It is also an ‘extra burden’ on schools and teachers in terms of time and
money. ‘The timing of the updates and changes is not appropriate — it is too late. It’s
changing the goalposts. We start teaching in June and then there is a change to Courses in
October/November.’ ‘SQA put things out far too late.’ ‘Late changes shouldn’t be reflected in
the QP (for that academic year).’
Understanding Standards (US) events were viewed as being well received and staff have
given very positive feedback about them. ‘I think the Understanding Standards are the most
popular.’ ‘Staff really appreciate the opportunity to sit down and have that professional
26
dialogue.’ ‘The networking is invaluable.’
However, in some schools SMTs raised the issue of a variance in the quality and
effectiveness of some events, with staff reporting the events were ‘OK’ but not reassuring
across the board and there is room for improvement. Information can often be contradictory
to that which is published and the approach taken is ‘changing by the minute’. Some
participants advised the first time that attendees learned about changes to Courses was at
US events. ‘In Understanding Standards the marking part is useful and maybe not a lot more.
There are nuggets of information that come out from Qualifications Managers. You get mixed
reports.’
There were also comments about the events being too late, the distance some teachers had
to travel as they were not able to secure a place at their nearest event, and that it is difficult
to secure a place at all at these events. There was feedback that one curriculum area
(sciences) had all their US events on the same day, so three teachers in one department
were out of school on the same day. Smaller schools, and schools with single-teacher
departments, find it more difficult to release staff for such events.
Many SMTs indicated it is very difficult to get teacher cover, including general supply, to
release staff for US events, training and marking. This was highlighted as being particularly
challenging in single-teacher departments. There were suggestions that some US events
were scheduled for weekends and that events are more spread out. SMTs indicated that,
whilst teacher cover is costly, they would encourage US events to continue and, whilst
publication of US materials is useful, it is better to attend an event.
There was some feedback that some subjects lacked exemplification material.
There are mixed messages about the success of nominees cascading information to, and
communicating with, colleagues within LEAs. Some LEAs have subject collaboration days
and these are useful. Not all LEAs appear to have an authority-wide policy on the use of
nominees. There needs to be an expectation of proper dissemination of information to
practitioners. There are many local information networks, which are becoming better, and the
use of subject facebook groups was mentioned as being one of the most useful sources of
up-to-date information. Nominees and SQA markers are being used in schools to support
judgements for Unit assessment.
SCIS supports SMTs to ‘plug nominee gaps’ in schools within the independent sector and
has assisted schools in having professional discussion amongst nominees.
There appears to be confusion and issues with prior verified materials. The process for
having materials prior verified was seen as being very lengthy and messages issued by SQA
confusing.
Course reports were seen by some as being very useful and are highly valued by some.
Many SMTs commented on positive visits to their schools by Subject Implementation
Managers.
27
8 Is there anything else you’d like to tell us about your experience of the new NQs
or any other SQA qualifications?
National 4
There was a general concern around the lack of an external assessment at National 4 and
valuing pupils. As it has no external examination, National 4 is perceived by some as being
of low value and currency. SQA needs to be aware of the strength of feeling on this matter
and consider whether it is caused by a misunderstanding, a miscommunication or a real fault
line in what we were asked to do by management board.
The following information is a list of comments about National 4 assessment from individual
SMTs:
‘Have you heard of #Nat4scum? … An N4 exam would give pupils parity and prepare them for N5. I don’t think we would want to argue for a reversal to what we had before and we don’t want to throw the baby out with the bathwater. The Added Value Unit is valuable for progression. We still need to have something in the examination diet with an element of the qualifications that has been assessed externally. This would provide focus for N4 students towards the diet, rather than the ‘padding’ for N4 students.’
‘The National 4 Course children feel devalued and, how the young people perceive themselves is important.’
‘A number of candidates are isolated and feel differently as they are doing N4. It’s seen as a fallback as opposed to achieving.’
‘Is National 4 going to be re-looked at? There is a stigma and pupils don’t value it.’ ‘We are doing National 4 students a huge disservice as there is no external exam … therefore it is devalued by those who sit it as well as N5 students who are doing the N4 Added Value Unit along the way.’
There is general concern around the lack of an external assessment at N4 level.
There was a general feeling that National 4 is undervalued by staff, parents and pupils, and that this qualification should be externally assessed.
The perception of National 4 within the school, and from parents, is that N4 seems less important as pupils are not doing an exam. Some families have a poor perception of Nat 4. There is a mixed perception from students — some see it as a stepping stone to National 5 in S5.
There was a general feeling of the SMT that some more able pupils would prefer doing an N4 qualification in order to avoid doing an exam.
‘In National 4, learning and teacher helps motivate pupils. Exams are not for everyone. I can see that no exam is a positive step for National 4. I think employers are beginning to understand this. It needs time to bed-in. The most difficult person to get the message across to is the parents. It’s perception led.’
‘I wouldn’t do that (have a National 4 exam). I would concentrate on getting some National 5 Unit passes. It’s about all pupils — about getting them currency and employment opportunities.’
N4 is pass/fail — maybe should be graded. No problem with internal assessment but grading rewards effort and gives a better indication of likelihood of success. ‘The kid gets the same thing (a pass) if they have worked their socks off or done nothing.’
Pupils achieving National 4 struggle to cope with the structure of N5 qualifications, particularly exams. The SMT is comfortable with no external assessment at N4 and would not like to see exams introduced at this level.
There is evidence that some LEAs have programmes for N4 pupils when the NQ
28
examination diet takes place and these are set up authority-wide for all schools within the
LEA.
Literacy levels in courses
Lifeskills Maths is ‘too wordy’.
‘Lifeskills Maths was a disaster for us. I think it was the amount of literacy required to get to
the mathematical content in the questions. In 2014 it was better. In 2015 there were so many
more words in the questions. We have withdrawn Lifeskills Maths from the school.’
SMT expressed some concern about N5 Lifeskills Maths. The Course requires a high degree
of literacy, which makes it very difficult for candidates.
SQA website and communication
A number of SMTs commented on the SQA website as being ‘difficult to navigate’, ‘difficult to
find information and ‘needs de-cluttered.’ ‘Trying to find things is a nightmare’ and a number
of respondents indicated it was easier to ‘Google’ to try and find things than navigate the
website. Staff are having difficulty finding core information that is up-to-date. Generally,
feedback suggests there are far too many updates, far too often. There was a suggestion
that SQA use apps for alerts for specific subjects. ‘It would be helpful moving forward in the
future, rather than wading through other subjects to get to theirs.’ ‘A lot of people tried to use
My Alerts, which historically didn’t work, so it’s difficult to get people back to this.’
National 5 Physical Education (PE)
A number of SMTs expressed concern about the absence of an external examination in
National 5 PE.
Post Results Service
A significant number of schools felt that the Post Results Service is a major issue with a
number describing it as ‘a lottery’ and ‘a minefield’.
29
4.7 Summary of teacher focus group responses
1 Thinking about your own subjects:
a) How well do you feel the pace and content of learning in S1-3 is preparing candidates for S4 and beyond — in terms of both knowledge and skills and attitude to learning?
Teachers felt that the pace and content of learning in S1-3 has been varied across subjects and, as such, was not fast enough. It therefore, in some areas, did not prepare pupils well enough for the senior phase. The result is that many schools are reviewing their broad general education (BGE). This will include starting the development of skills earlier in the BGE and starting some work on content towards Nat 4/5 in S3. As well as the pace and content of learning in S1-3 varying across subjects, a number of teachers expressed the view that the BGE was not preparing candidates very well for work in the senior phase and because of this, the commencing of work in S3 was felt to be essential to allow sufficient time to cover courses. If not, there was a time issue, especially in S4, to deliver courses adequately. In doing this, the majority of teachers felt that BGE level 4 Experiences and Outcomes were flexible enough, for them to reasonably include content for the qualifications and still cover the concepts important to the delivery of the BGE.
b) How smooth is the progression in demand between the different levels of a subject?
Teachers felt that progression in demand between different levels of a subject was not smooth and was variable depending on subject. The high volume of content is an issue in some subjects. This was indicated in a number of centres — especially in subjects such as Maths, Sciences and History. Teachers also felt that there was a huge jump in the ‘step up’ from National 4 to National 5, but less so from National 5 to Higher. Teachers felt that there is also now the additional pressure on their time due to introduction of coursework assessments, eg assignments, across subjects, in addition to a question paper. In many cases, National 4 was not seen as valuable, because there is no external exam. This also leads to problems of motivation for lower ability pupils. There was, therefore, a fairly widespread request by some teachers, to introduce a National 4 external exam. Some teachers would prefer the National 4 Added Value Unit (AVU) to take the more formal format of an exam. Many teachers carry out the AVU in S3 with their pupils. Many teachers expressed the view that bi-/tri-level teaching is problematic in some subjects, especially knowledge-based subjects.
30
2 SQA exemplified three different approaches to Unit assessment — Combined, Unit-
by-Unit and Portfolio: a) What do you feel are the advantages and disadvantages of each approach for your
subjects?
The majority of teachers are teaching and assessing using the Unit-by-Unit model, as it is easier to deliver and record, even though they felt it was very time-consuming. However, there are several challenges regarding Unit assessment including the time taken up for assessment and re-assessment. Also, there are issues of pupil availability for assessment, ie pupils not turning up or being absent on the day of the Unit assessment. However, teachers also noted that with increasing teacher confidence and familiarity with courses, this was allowing them to explore different approaches to Unit assessment eg the Portfolio or Combined models. Some used different models with different levels of qualification in their subject. Also, the majority of teachers felt that there are too many Outcomes and Assessment Standards in each Unit for candidates to overtake, and linked to that is the issue of assessing, tracking and recording them. This leads to too much time spent assessing and thus they feel that the process of marking Unit assessments is very time-consuming and bureaucratic. Some teachers therefore suggested that cut-off/threshold scores should be introduced. The majority of teachers felt that there was a pressure of assessments and re-assessments in various subjects, squeezed into the same timeframe, and that this was leading to serious workload implications across subjects. Some teachers suggested that Course assessments could help candidates overtake Unit assessments, ie if a candidate achieves the course there should be no need for them to be assessed separately for the Units. The numerous changes to Unit assessment support materials are causing confusion and anxiety in many subject areas. Many teachers expressed the view that open-book assessments are of little value and do not have enough rigour, compared to the demands on candidates when performing under examination conditions without notes and against the clock. Finally, the majority of teachers indicated that the questions/tasks in the Unit assessment support materials are too simple or do not have the appropriate level of demand; therefore many teachers give additional A/B tests, which is leading to ‘assessment inflation’ for candidates in some subjects. The prior verification process can be very time-consuming and so was an issue in some centres.
31
3 NQs include Unit assessment as well as Course assessment (N4 Added Value Unit,
question papers and coursework)
a) What is your experience of balancing the work and assessment of the Units with preparation for and undertaking the Course assessment?
In general terms, most staff felt that:
the three parts of the course ARE NOT in relative balance
parts of the course are more squeezed for time than others
pupils are NOT sitting assessments in these areas when they are ready It was felt that the CfE senior phase is assessment-driven, and that Unit assessment is a ‘tick box’ exercise that feels like ‘jumping through hoops’ for assessment and verification. Many teachers expressed the view that time spent on Unit assessment is far greater than its value and that, conversely, the Unit assessment level of demand is too low and its value is arguable. Concerns were raised by some teachers regarding no differentiation in Unit passes, eg ‘performances by candidates range from excellent to barely pass!’ Some teachers felt that they require more specific marking instructions. Many teachers stated that pupils could be doing a large number of Unit assessments during the year across all their subjects, and that the paperwork re marking, tracking and recording of Assessment Standards has led to increased workload and time issues. Some teachers had, in the early years made whole classes complete the National 4 AVU. In many schools, the National 5 borderline candidates were also being asked to do the National 4 AVU. Some teachers were photocopying all National 5 assignments before sending them to SQA and then marked for National 4 AVU, if necessary. Some teachers felt that the Unit assessments and Course assessments are disjointed. The issue of delivery of bi-, tri- and multi-level classes was raised. This was especially true in subjects which are very ‘content-heavy’ and different content at different levels, eg Computing, Physics, Biology, etc. Some staff commented that pupils were asked to do a lot of coursework components across all their subjects. Several pupils have commented to their teachers, ‘Not another assignment ...’ Some teachers suggested that achievement in Course assessment should overtake achievements in Unit assessments. There were several requests from teachers to introduce a question paper for National 5 Physical Education and from some to drop Listening & Talking from English at National 5 and Higher. Some teachers suggested the idea of introducing interdisciplinary assignments, to reduce the assessment load on candidates.
32
4 Schools are adopting different models for their delivery of the new NQs:
a) Is any work on NQs being done in S3 and, if so, what impact has this had on your candidates?
b) Have any of your pupils undertaken the N4 Added Value Unit in the same year as the
N5 assessment for that subject and, if so, are you continuing with that approach?
The majority of teachers stated that their original approach in Year 1 was to have all N5 candidates sit the N4 AVU. The majority of teachers now have only some National 5 candidates doing the AVU, whereas previously there were more (see responses to Q3a). In some centres, only pupils identified as National 4 level/borderline do the National 4 AVU. Many centres are doing NQ work in S3 and ‘banking’ it. In some centres, there is some work being carried out in S3, but no formal Unit assessments. In others, Unit assessments are being done in S3. The only subjects not doing this tend to be subjects which have a lower candidate uptake or those not being taught at lower levels. However, in some schools, they can’t do assessment in S3, due to timetabling constraints, as they can come back to the subject in S4. In some schools and for some subjects, all pupils still do National 4 AVU. Many teachers thought that it was unrealistic for courses to be done in one year — need 18 months to two years. However in some centres, they start National 5 in S3, therefore they feel they manage time for the courses more adequately. Some staff recommend that a two-year Higher would be more appropriate for some candidates, who need more time. Some felt that there was very little articulation in English between National 4 AVU and National 5. Some staff suggested that the Scottish Government should specify the number of subjects candidates should take in an academic year, ie define a standard curriculum model, between schools/colleges. In a few subjects, in a few schools, the National 4 Course is completed in S2, though not certificated.
4 The documents for the new NQs were published in sequence for each subject and
level, from Course and Unit specifications to Unit and Course assessment support. a) Which documents have been of most use to you?
The documents of most use are (in order of priority):
Course and Unit support notes (CUSN)
Unit assessment support packs (UASPs)
Course assessment specifications (CAS)
Course specifications
Unit specifications
Verification and course reports
Specimen question papers (SQP)/Exemplar question papers (EQP)
Use of past papers and marking instructions (MIs)
33
b) Now that we are much further into the implementation cycle, would you welcome
some rationalisation of the documents?
Everyone stated that a rationalisation of documents was required and that there was too much duplicated/irrelevant information in documents. Many changes to documents come far too late in the year. Teachers would prefer that any changes are made once a year in May/June only, and any further changes held back until the next year. Teachers are finding it very difficult to keep up with the high volume of updates to documents, and late changes and high volume of updates in October are not helpful. The majority of teachers thought that the SQA website was difficult to navigate. Some felt that UAS materials and course specifications are of a poor quality. Some published documents still had ‘track changes’. Sub-component marks would be useful in some subjects. One centre felt that the CUSNs need to be more ‘visual’ for Art.
6 SQA support for the implementation of the new NQs includes the wider role of
nominees, verification reports, Understanding Standards events and course reports.
a) How useful have these been to you?
Wider SQA support has impacted on centres to a high degree. Most teachers found Understanding Standards/LEA events useful, although they should be scheduled earlier in the academic year. More exemplification to documents eg question papers and coursework assignments is required. There is also an issue regarding availability of places and ‘usability’ of the booking system. It was felt by teachers in most centres that there is a need for more events. It was evident from discussions with teachers that not all nominees were aware of their responsibility to cascade relevant information to other teachers within their centre/authority. Teachers who were also nominees felt that the benefits were variable; some felt it gave them a better understanding of standards. The majority of teachers thought that SQA nominee trainings were very good, although this opinion varied across centres and subjects. There was mixed feedback about verification and course reports: some were very useful but many did not contain enough detail. Visiting verification and access to the Subject Implementation Managers (SIMs) were considered to be very useful, although some were lacking in ‘quality, consistency and professionalism’. Many teachers stated that there were massive CPD issues in some subjects eg Graphic Communication, Design & Manufacture. This is especially evident when courses have been amalgamated and therefore there has been a significant change to the course content.
34
7 The new NQs seek to encourage a different way of learning and teaching in the
senior phase (as in the broad general education).
a) What is the impact of personalisation and choice, and of opportunities for independent learning and self-reflection?
Owing to time constraints, not all centres were asked this question. The majority of staff asked thought that there was independent learning. A small majority of staff thought that pupils reflect on their own learning. The opinion of staff was equally split regarding the new NQs encouraging a different way of learning and teaching in the senior phase as well as in broad general education. The majority of staff thought that there were differences between subjects and a small majority of staff thought that personalisation and choice in the senior phase was appropriate. There were many positive responses regarding the opportunities for personalisation and choice in subjects such as Music, AH Biology and Physical Education. However, a large minority of teachers felt that personalisation and choice was restricted to projects/assignments and, in some subjects, the teachers stated that personalisation and choice is limited/restricted due to lack of resources.
8 Is there anything else you’d like to tell us about your experience of the new NQs or
any other SQA qualifications?
This section is intended to capture ‘other issues’ important to the group:
Over assessment at Unit level across subjects is a consistent issue.
There are issues over quality of verification in some cases (see Q6a above).
Physical Education teachers expressed concern over the lack of a question paper in National 5 Physical Education, and the effect this has on the teaching of bi-level National 5/Higher classes.
Skills for Work Courses: The criteria for approval for these are too vague. The process involves too much paperwork.
Some teachers expressed concern over the expected level of English/literacy being too high in many question papers (eg Higher Geography; Higher Biology; Higher Maths).
Science — ‘exciting’/‘progression much better’/‘context-based questions are good’.
Maths sub-skills — it was indicated by Maths teachers that these are problematic when it comes to assessment.
Many teachers stated that National 4 requires an external assessment, although they recognised this may disadvantage lower ability pupils (National 3/4).
35
4.8 Summary of candidate focus group responses
1 You are all in the senior phase — S4 and above:
a) Thinking about what you learned in terms of skills and knowledge in S1-S3, how well did
that prepare you for your work in the senior phase?
High level message: Not very well, but the experience varied a bit between subjects. Most candidates had difficulty with S3-S4 transition as ‘it was a massive jump’ and they felt the pressure of the step-up. Partly this was because S1-3 was ‘so easy’ and the skills taught were basic so they ‘didn’t learn much’ and therefore it was ‘pointless’, ‘irrelevant’ and a ‘complete waste of time’ as they were ‘not working towards anything’ and teaching was ‘all over the place’. A number of candidates commented that they ‘don’t remember much’ about what they learned, maybe because they ‘didn’t listen back then’ but also because it wasn’t challenging. In terms of basic skills in subjects like Maths and in more practical subjects such as Art the candidates felt a bit more prepared, but in other subjects such as English candidates felt they were ‘chucked in at the deep end’ with their first essay. ‘In 3rd year there is time for life-experience stuff. You lose this in 4th and 5th year: you didn’t have time to stop and think where you could use this in life’. In terms of exams, for some candidates it was a ‘massive jump’ from broad general education (BGE) to senior phase (SP): ‘Junior phase felt more involved: SP feels like everyone is just writing notes’ and ‘drowning in assessment’. A number of candidates said that they should have received more preparation for exams in 3rd year (such as the way in the way questions have to be answered) and how to write essays: ‘S1-3 didn’t help me prepare to meet deadlines or for the nature of assessment’. Another challenge for some candidates was that they went straight into an N5 course in S4. Although some candidates felt that S3 was better preparation than S1-2 and was a ‘good taster’ (they knew ‘a lot of the basics and were building on the foundations’ in some subjects such as Humanities, Languages, Maths, Science), many candidates felt that they lacked key knowledge or skills when starting S4 and were unsure of what some specific subjects actually involved. Other factors that were different between BGE and SP included the increase in content/workload, the amount of homework and the greater independence of learning. In a number of schools there was a feeling that more support (including S3 exams) was now available for candidates in S1-S3 and that this provided better preparation for senior phase.
b) Did the speed of learning in S1-3 prepare you for the work in the senior phase?
High level message: No
S1-3 was described as ‘slow/relaxed’ and ‘taking it easy’ which ‘didn’t prepare [us] at all in terms of
pace’, making it a big ‘step up’ to S4, although some candidates felt this pace made it manageable. A
number of candidates mentioned that the pace increased in S3 compared to S1-2. Many felt there was
not enough time in senior phase to undertake Unit assessment and prepare for Course assessment:
many candidates felt that they were ‘rushing everything’ and had to ‘cram things into S4’. Some
candidates felt that the pace of learning in S1-3 has improved since the first year of implementation.
36
2 Thinking about the qualifications you’re doing this year:
a) How is the step up from what you were doing last year?
High level message:
National 4–National 5: although some found it a challenge, N4 was really easy for most
candidates, so the move to N5 was a big step (‘too much’; ‘thrown in there’), particularly with
the amount of writing involved and having to sit an exam for the first time. However, the
transition from N4-5 was straightforward for some. We did not speak to many candidates who
were at N3 or N4 level only.
National 5–Higher: straightforward for some subjects but more difficult in most subjects.
Workload, content, complexity, independent learning and pressure increased a lot from N5 to
Higher — ‘in school and at home, no time to go over things because of the volume and pace of
work’.
Higher–Advanced Higher: straightforward in some, but difficult in other subjects. Big step: a
lot more content, faster pace, reading and independent study:
There were a lot of specific subject comments, but these were not mentioned often enough to
draw attention to particular subjects — apart from the ones mentioned below.
N4 was seen by some to be ‘too relaxed’ because of a lack of an exam at the end and the perception of N4 was generally negative: ‘N4 regarded as not important’. The step up from N4-5 was ‘very difficult because of the change in pace and pressures’. N4 was seen as ‘not as academic and the volume of work was different’. Some candidates did N4 and N5 in S4 and found that difficult, especially the ones who dropped down to N4 because they weren’t at N5 level. Moving from N5 to Higher was often seen to be a ‘big jump’, but the jump can be made ‘if you work hard’. One of the key differences was the amount of content and associated workload. There were seen to be ‘huge’ differences in difficulty across subjects, which made it difficult to choose subjects when candidates didn’t really know what would be involved. The step up from N5-Higher PE was specifically mentioned a number of times because one of the Course components is different (Portfolio at N5; Exam at Higher). In some subjects candidates felt they were studying the same thing over two years, but in more detail at Higher, which made it ‘good for the exam but you lose interest in the 2nd year’. Essay-writing was also specifically mentioned as ‘a big jump’ in social subjects. However, there was also a realisation that ‘there is supposed to be a gap — they are different levels’. Quite a lot of candidates were ‘crashing’ Highers and this also caused specific issues. There was mixed views about the move from Higher to Advanced higher (AH): some candidates felt it was straightforward in some subjects but others felt it was a big step up, although in many cases this was expected: AH requires a lot of independent learning and ‘you have to get on with it yourself’. Although this is to be expected and was seen as stressful at times, the challenge and pressure was also enjoyed by some. Problems encountered at AH included a shortage of — or problems with access to — teachers and having to do courses online: ‘Could have been a better transition from Higher to Advanced Higher’; ‘just thrown into independent learning; ‘too much change at once’. The emphasis in AH was ‘more on dissertations and reports, so the intensity and pressure of exams in S5 has shifted to other skills’. In general, candidates felt that AH was a good preparation for university. Particular issues arising from lack of progression were experienced by candidates who did N5 then the old Higher, or the old Higher and new AH, as the content and structure varied.
37
b) Did you do any work towards your qualifications in S3 in any of your subjects (eg Unit
assessments)?
High level message: Varies considerably between schools.
It varied between schools whether teachers in all, some or no subjects undertook work towards senior phase in S3. Candidates who started some subjects in S3 commented that it was ‘easier’ to do this and a number mentioned that they would like to have at least started N4 in S3 but that their teachers weren’t allowed to do that. Quite often it was done only in specific subjects. In some schools, this was done to assess whether candidates should be placed into N4 or N5 whilst in other schools, teachers started teaching Units or coursework. In some centres, it was because they had ‘got through N4 so quick that they could do N5 prep in S3’. It also varied greatly between schools whether all, some or no subjects required candidates to undertake the AVU in S3. Some schools gave some or all of the candidates the AVU in S4, and banked it as a fallback in case they failed N5, although not all candidates were aware that it would be used for this purpose. ‘You were seen as an individual in S1-3. It wasn’t like that in S4 and S5.’
3 Now think about the work you did last year.
a) What was your experience of assessment during last year?
High level message: Unit assessments took up a lot of time and didn’t leave enough time for
work on the coursework component and preparation for the question paper: ‘too much
assessment and not enough time’ meant that assessment was ‘very draining’. Candidates
weren’t always sure what was required of them in Unit or Course assessment and teachers
didn’t always seem to clear of the standard expected.
Many candidates got Unit assessment throughout the year ‘just after we learn it’ which meant
there was often a clash with other subjects, but for others it was ‘random’ and ‘episodic’; some
felt ‘unprepared because they hadn’t had enough notice, which put them under quite a lot of
pressure'. It was felt by many that there was too much emphasis on Unit assessments rather
than learning and teaching. They were also seen by many as ‘just a drag’ because there was
so much other work to do (prelims, assignments and exam prep). Many mentioned that it was
‘stressful’ and ‘annoying’ that they had to meet all the Assessment Standards and that the level
of difficulty in Unit assessments varied between subjects. It was felt by some that because they
were doing Unit assessments just to pass, that there ‘was no real incentive. In some subjects
(eg Sciences) the assessments were ‘okay’ but in other subjects (eg Geography) they were
quite vague as to what was required and candidates think this also made it hard for teachers to
mark them. Vagueness also made it difficult for candidates to ‘know how much to write’ and
what was expected’. A lot of candidates still referred to Unit assessments as ‘NABs’ — this
may be terminology inherited from teachers, but in some cases they had been doing old
Highers that included NABs.
With regard to coursework assignments, some candidates felt that initially in some subjects
there was a lack of clarity around the format and that they didn’t get any guidance from
teachers. Especially at N5, there was not enough past paper type documents available to allow
38
proper preparation for the question paper component. Some N5 candidates mentioned they felt
stressed about their exams because they hadn’t done one before. A number of candidates felt
that assessments were a test of how much or how quickly they could write rather than a test of
their skills or knowledge: ‘I’m not a fast writer: I can’t speed write, but there’s just so much to
write. I knew the course inside out, but just couldn’t write fast enough’.
b) How much of your time did it seem to take up over the year?
High level message: Responses varied, but many felt there was too much assessment,
especially at bottleneck times. Many candidates mentioned the negative impact of assessment
on their out-of-school activities.
Many candidates felt that there was less focus on learning and teaching as it was just
‘assessment after assessment’, but there was a feeling in some schools that teachers are
better organised this year. Many candidates felt the volume of assessment was too much (‘it
felt like drowning’) and consequently they felt under a lot of pressure, especially at the end of
Units and between January and March, as they didn’t have enough time to prepare for Course
assignments as well as Unit assessments and therefore weren’t able to perform at their best
level. One candidate mentioned that there was so much assessment that the ‘fun was taken
out of the subjects’. Although a number of candidates felt that Unit assessment took up so
much time that they didn’t get much time for Course assessment, some candidates felt that
assessment was well spread out. A number of candidates mentioned that extra-curricular and
out-of-school activities and p/t work suffered because school work and revision took up so
much time: ‘I felt I had no time to do what I wanted to do’ and ‘Teachers expect that you have
no life outside of school’.
4 Unit assessments are designed to help you prepare for Course assessment:
a) How do you feel the Unit assessments helped you to prepare for course assessment
(that is for question paper and coursework)?
High level message: Not at all well. Most candidates felt that there was no point in Unit assessments if they didn’t help prepare for the Course assessment (particularly in terms of achieving high grades), give an indication of what the exam will look like or let them know how well they are going to do in the exam. Most think it would be better if they were ‘a smaller version of the exam’. Most candidates felt that Unit assessment was ‘very easy’ and not pitched at a high enough level and therefore a ‘waste of time’. They don’t help prepare for Course assessment and this gave a ‘false idea of the standard needed’ for the exam, ‘resulting in a ‘deceiving perception of their level’, ’unclear perception of their progress’ which gave a ‘false sense of security’. Some candidates felt that Unit assessments ‘give basic understanding’ and were helpful in some subjects, but still commented that ‘there was just too many of them’. One group said that ‘the level of difficulty [of Unit assessments] was appropriate at the beginning of the year, when they were unfamiliar with the Course, but not as they got towards Course assessment’. The question paper (QP) was often seen as more difficult than the Units. The difference in the wording of questions was specifically mentioned by a lot of candidates: the Unit assessment
39
questions are ‘nothing like the exam at all’. Many teachers provide ‘A/B’ tests to help prepare candidates for the QP and give them a more accurate picture of their progression, as Units only give an indication of a ‘C’ level pass and so are not a preparation for the Course. Some candidates felt they were being asked to ‘dumb themselves down’ and were being told by teachers ‘not to overthink’ their answers to Unit assessments, which just have to be done as a ‘formality’ and was seen as ‘just like jumping through hoops to get to the exam’. Some felt that the answers they had to provide were formulaic and others queried why they were assessed on things that weren’t covered in Course assessment. Many candidates felt that having Unit assessments as open book and with no time limit was unhelpful preparation for the QP and it would be better if it was closed book as then they ‘would get a feel for what the exam was like’. A number of candidates mentioned that they would like Unit assessment to be harder and to have marks so it would give them an indication of how much to write and how they are progressing. Some queried why they had to do Unit assessments at all as they were ‘pointless’ and felt that ‘past papers’ and prelims were more useful in understanding the standard and preparing for the QP. Others felt that in some subjects, ‘it wasn’t testing if you can understand the subject content’: it was an issue of ‘how much you could write in a small space of time’, just a test of what you can memorise or formulaic/artificial, in that you had to write down the exact thing in the required structure or you didn’t pass: ‘Unit assessments don’t feel like learning — they feel like regurgitation’. Some candidates felt that the non-QP component is ‘good, because you know if you do well in the assignment you feel under less pressure for the exam’. Other candidates felt that teachers don’t put enough time and effort into the coursework even thought ‘it can bring your score up’. For some, the assignments in particular ‘felt like a chore’ and were ‘too time-consuming, but for other candidates coursework (including assignments) was liked because ‘it’s not just about what you do on the day’ and ‘it’s something different’. It also meant that ‘if you do well in the assignment, you feel under less pressure for the exam’. Another candidate mentioned that ‘assignments are good as you are going to have write papers in the world of work’. A number of candidates queried the weighting given to course components and felt the non-QP component should be higher for practical subjects but some didn’t know what the respective weightings were. However, some candidates said that Unit assessments are necessary and good to have throughout the year, as it helps them revise as they go along and once they’ve done all the Units they link together and help with their coursework. Some candidates felt that in some subjects Unit assessment did prepare for coursework (English, Computing Science), but not necessarily for the QP component.
b) If you were re-assessed for any Unit or part of a Unit, how did that happen?
High level message: Generally, re-assessment took place outside class time and was often seen as ‘pointless’ or ‘demoralising’, but some found the identification of areas they needed to work on useful. There were differences between schools and between departments within a school about how re-assessment was organised. Candidate experience of re-assessment was school- and subject-dependent. It was often seen as ‘pointless’ (‘you have to rewrite a whole question to write a sentence to pass’; ‘the difference of one sentence gets you a pass) or ‘demoralising’, as you can get good marks but still have to re-sit one particular aspect, which in some cases was ‘just a case of adding some information to a question’: ‘even clever students are failing assessments because they miss one word. We are failing on exam technique and language’. Some candidates queried why they had to get everything correct for the Units, but could pass the Course with 50%.
40
However, some candidates said that re-assessment can be good, ‘if you only got one bit wrong, you’d only have to re-do that part’ and it shows candidates what key areas you have to work on, but logistically it was difficult for teachers to manage and candidates often have to go in at lunchtime or after school. A few candidates felt that re-assessment was organised well, but many felt that it was ‘rushed’ and crammed into short periods alongside planned assessments and that they were ‘constantly catching-up’. A number of candidates commented on the amount of class (or free) time lost to re-assessment. Also, in some cases, re-assessment was seen as harder than the assessment: ‘there was a very rigid interpretation of the AS in the UASPs’ and candidates were asked to re-sit the whole UASP again, not just the parts they had not achieved’. Some candidates pointed out that it was easier to do re-sits in S5 because they take fewer subjects than S4 and therefore have more time in class for re-assessment. There were particular issues with catching up on assessment and re-assessment if the candidate had been absent.
5 Now think about the subjects you are doing this year, or those you did last year.
a) How much personalisation and choice do you feel there is in how you’re learning and
what effect does that have?
High level message: Varies significantly between subjects and levels.
Some candidates felt that choosing their own topic ‘can tell teachers more about you and helps
the teacher get to know you’ and ‘shows the teacher that you’re putting effort into it’. ‘It’s like a
breath of fresh air’ because it ‘means you get to do something you’re interested in and you’ll do
a better job of it’ and it’s therefore ‘more enjoyable to work on’. Others preferred ‘teachers
telling them which topics to do’ and felt that choice makes it harder if they’re not good at
making decisions: most candidates chose a topic from within the Course content anyway
because they were ‘more confident’ about it and the teachers found it ‘easier to mark’.
Teachers often gave candidates a ‘guided choice’, ie presented them with a small number of
options/suggestions: sometimes only two. One candidate said ‘The teacher put up four
different topics with the average marks that past students had achieved and where the highest
marks had been and this helped steer us’. Another said that ‘the teacher asked me to modify
my question so that I could write more about it and might be able to get more marks and that
was fair enough’. Another said ‘personalisation versus good marks — candidates go for best
marks option every time.
However, when candidates weren’t allowed choice, some felt it was ‘demotivating’,
‘unappealing’, ‘prescriptive and so boring’ and that ‘no choice makes learning quite difficult’. In
general it was felt that the higher the SCQF level, the more choice they were given. Between
subjects, there was more choice in Languages, Humanities and HWB subjects, because there
were often resource and practical limitations to choice (eg in Science where experiments were
involved). Some candidates said that there was more choice this year due to an increase in
teachers’ confidence and understanding of the courses, whereas other teachers had reduced
choice because of the amount of marking they had previously when candidates did different
topics, or because ‘in other schools, where they did it all together, they got better marks’.
Candidates mentioned implications for choice in terms of the amount of subjects they studied
41
which varied from 5-8 in S4: ‘In 3rd year it is very hard to pick from 13 subjects to five in S4. I
ended up picking the subjects I was good at because I didn’t know what I wanted to study at
university’; ‘seven subjects in S4 is too much’; ‘we did eight subjects last year — a lot of extra
work — unnecessary’. Some candidates mentioned that there was less personalisation and
choice in the senior phase compared to BGE and that in some cases this was because there
was ‘so much content in the courses that there was little time for interaction between the
candidate and teacher and little time to consolidate’: ‘I don’t feel like I’m an individual from S4
onwards’.
b) How much responsibility do you feel for your own learning?
High level message: Varies between subjects and levels.
Candidates felt responsible for their own learning if they chose their own topic, as it ‘makes you
want to work at it’ and ‘stick at it more’. In AH there is a lot of independent learning as they
often have limited contact with their teacher. Some would prefer more support and guidance
but in general they understand that this is good preparation for university. Some candidates
said, especially at Higher and AH, that ‘teachers give you what you need and it’s up to you how
you use it to prepare for exams’ and ‘it’s up to you whether you make the most of the
resources’. Candidates clearly thought qualifications were important and would impact on their
adult life.
However, some N5 and Higher students felt that because classes are big and there is so much
content, ‘the teacher just rushes through everything and tells candidates to study at home and
ask for help during supported study’, so there was stress associated with this type of
responsibility for their learning. Other candidates said that because they ‘had to fill in the gaps
yourself’ and ‘everything in class is at C’ that they needed to work at home. Some mentioned
the difference between homework and classwork you have to finish at home’ because there’s
not enough time to cover it in the class. Many candidates mentioned the problems arising from
lack of support materials to help them study: not enough practice/past papers, no BBC Bitesize
in their subject, textbooks not available till April etc.
c) Do you usually have a clear idea of how well you are doing?
High level message: Varies between centres, subjects and levels.
Candidates indicated that they received a wide variety of feedback, ranging from specific
useful comments (eg about Unit assessments, essays and prelims) to no feedback at all —
except whether they had passed or not. It was often subject-dependent, and candidates felt it
was often just because the teacher was too busy to give detailed feedback: ‘Your parents get
feedback at parent’s night and that’s half way through the year: it’s too late’. In some cases (in
bi-level classes) candidates weren’t aware of what SCQF level they were at: ‘I wasn’t sure
what I was doing in S3: I didn’t know if we were working on N5’; ‘I don’t know if I have N4 in
Graph Comm. No-one really knew if they were doing N5 in the top set’. Many students didn’t
42
know what grade they were performing at because no marks are given in Unit assessments:
‘You know if you’re failing, but not how well you’re doing’; ‘don’t know how well you’re going to
do — does anything link?’. This was harder to know in relation to more subjective subjects, and
less of a problem in ‘high content’ subjects. Candidates felt it was useful when marking
instructions were shared with them, but even then it wasn’t always clear where — or why —
the marks were being allocated. Bi-level or multi-level class organisation sometimes had a
negative impact on the level of support received, particularly for N4 candidates.
Some candidates would have preferred regular testing to help them gauge where they were
and to help them prepare for the exam. This already happened in some centres in subjects
such as Maths, Science and History. In some schools tracking reports were used (replacing
report cards), but it ‘doesn’t help you know if you need to work harder: It doesn’t give any
detail. It just says ‘on track’. Some candidates felt the ‘teacher doesn’t really fully know —
couldn’t tell from Unit assessments’, but others felt that the progress reports they get from
teachers helped them understand what they had to achieve. A number of candidates said they
thought they were doing well, but didn’t do well in the exam or vice versa. This was raised as a
particular problem given that there is no longer the opportunity to appeal.
6 Is there anything else you’d like to say about your experience of the new NQs?
Many candidates raised issues that don’t fit into any of the categories above. These include:
The timing and amount of changes were a burden for candidates, particularly when Course assessment was affected
Many candidates felt that ‘teachers were learning too’ and didn’t always know what they were teaching and that there should have been more communication between SQA and schools, and more materials, so that teachers were better prepared and less pressured/stressed, although some departments were better prepared than others. The 6th-year cohort in particular (we were the ‘guinea pigs’ every year’) felt that this disadvantaged them. In general, candidates reported that teachers seem better prepared this year.
Many SQA documents weren’t easy for candidates to read — they were not clear and didn’t use plain English: too much ‘teacher speak’.
There were a number of issues related specifically to question papers: Are marking criteria being applied properly? The length and language in questions detracted from the subject content and was too complex at times; Exams don’t always appear to be testing the right things, eg there is too much English required for the Art QP/too much Maths needed to do Chemistry.
Some practical subjects (eg Drama) had too large a workload because they’ve been made too ‘academic’.
‘Is there any way you can make Maths easier?’
43
4.9 Statistical data — teachers (background questionnaires)
Total number of teacher questionnaires received 249
Demographics % of total
Male 120 48.2
Female 163 65.5
Note:
1. The total number of questionnaires is a count of the documents we collected 2. The male/female demographic was gathered from the seating plan in the Focus group Scribe forms
YES NO
Questions: Number % Number %
Do you teach vocational qualifications? 36 14.5 199 79.9
Are you an SQA appointee? 92 36.9 144 57.8
Are you an SQA nominee? 91 36.5 145 58.2
Attended implementation events (only) 33 13.3 n/a
Attended Understanding Standards events (only) 46 18.5
Attended both types of events (or none) 115 46.2 45 18.1
Have you developed prior verified material? 83 33.3 137 55.0
Do you use the N4 AVU for N5 candidates? 135 54.2 91 36.5
In SQA-produced Unit assessment support materials, three approaches to assessment have
been exemplified — Combined, Unit-by-Unit and Portfolio.
Question: What has been the most common approach in your subject(s)?
Unit assessment support pack
Unit-by-Unit Portfolio Combined Mixture
Number % Number % Number % Number %
144 57.8 13 5.2 29 11.6 45 18.1
0.0
20.0
40.0
60.0
80.0
1
Unit assessment support
Unit by Unit %
Portfolio %
Combined %
Mixture %
44
4.10 Statistical data — candidates (background questionnaires)
Total number of candidate questionnaires received 332
Current year group % of total
S5 147 44.0
S6 185 55.7
Question: On a scale of 1-5, rate how hard you had to work in S4? (1 would be not very
hard, 5 would be where you had time for little else, working at evenings and weekends.)
Response Number of responses % of total
1 7 2.1
2 30 9.0
3 106 31.9
4 120 36.1
5 55 16.6
Response Number of responses % of total
Then in S5? Less hard 38 11.4
The same 55 16.6
Harder 225 67.8
Response Number of responses % of total
Then in S6? Less hard 30 9.0
The same 58 17.5
Harder 88 26.5
Question: Have you used the SQA website to get information about your subjects? If so,
how easy was it to find what you wanted on a scale of 1-5 (1 being easy, 5 being very
difficult)?
Response Number of responses % of total
1 47 14.2
2 77 23.2
3 68 20.5
4 33 9.9
5 8 2.4
Have not used 56 16.9
Note:
1 Not all candidates handed in a completed questionnaire
2 Not all candidates responded to every question
45
Question: What information or support were
you looking for on the SQA website?
Question: Did you use other websites? If so,
which website(s)
Art & Design portfolio information BBC Bitesize
Data booklets Bright Red
Chemistry assignment information ChemWeb
Course notes/content Edmodo
Course specification Education Scotland
English course code eFlash Learning
English close reading Espanol Extra
Equation sheets Evans 2 ChemWeb
Exam timetable Fizzics
Grade boundaries GLOW
Marking instructions MathsRevision
MySQA Modernity Scotland
Past paper question grid Languages Online
Past papers RSPB Oronsay
Specimen question papers SCHOLAR
School website
Shmoop
SparkNotes
Wikispaces
YouTube
46
4.11 Online survey analysis and commentary Analysis of responses to an online survey of teachers in 40 Scottish secondary schools conducted during November and December 2015 Introduction In October 2009, an SQA Qualifications Governing Group (QGG) was established and, over the next few months, considered draft design principles. Design principles were approved by QGG on 15 January 2010 and subsequently signed off by the Curriculum for Excellence (CfE) Management Board. The bullet points below describe the model of course design and assessment approved by QGG. A new qualifications system that:
Supports the values, purposes and principles of CfE and supports the learning of the new curriculum, including its breadth
Provides a seamless transition from Outcomes and Experiences, with increased emphasis on skills
Is inclusive, coherent and easy to understand for pupils, parents, staff, employers and other users
Meets the needs of all learners in progressing from prior levels of achievement and provides opportunities for learners to develop at different rates, at different times, in different areas across the curriculum
Provide clear and smooth progression and articulation between different levels of qualifications, from Access to National 4 and 5, to Higher and AH, and on to post-school learning and employment
Involves an overall approach to assessment which reduces the time learners spend on assessment for certification and allows more time for learning, and more focus on skills and integration with other aspects of learning
Results in assessment that supports, motivates and challenges learners, with more scope for personalisation and choice
Maintains high standards, credibility and relevance In the third year of the new National Qualifications, during the first year of Advanced Higher implementation and the first year in which all schools were required to take the new Higher, SQA sought to survey staff in secondary schools across Scotland on the extent to which they considered that those objectives had been met. Materials and methods Creating the survey The complex bulleted statements that communicated the design principles were broken down into 25 simpler statements which could be responded to individually, relating to either the qualifications or their assessments. These statements are detailed in the first column of Table 1, below. Respondents were asked to rate their agreement with the presented statements on a 5-point Likert scale (Strongly agree; Agree; Neither agree nor disagree; Disagree; Strongly disagree). For a small number of relevant items, a ‘Don’t know’ option was also provided. In addition, two opportunities were provided to make additional comments on the new qualifications and on assessment for the new qualifications.
47
The survey design was approved by the Heads of Service for CfE and for Research, Policy, Statistics and Standards. The survey was reviewed after a small number of schools had completed it and on the basis of unexpected responses. One item (regarding the new qualifications being ‘inclusive’) was re-worded for improved clarity. (It was later reported that ‘Geography’ was missing from the list of subjects and this omission was remedied on 2nd December.) At the start of the survey, respondents were asked to select one or more subjects, as follows: ‘Your experience of the new qualifications may relate to all Curriculum for Excellence (CfE) subjects or only some. Please choose from the list below and consider only those subjects when responding to the survey questions that follow. My experience of CfE relates to…’. This statement was followed by a list of all subjects in which the new qualifications are offered at any level. This list included an ‘All subjects’ option to accommodate SMT respondents. Distributing the survey During November and December 2015, the CfE team conducted focus groups with candidates, teachers and members of the senior management team in 40 selected schools covering all local authority areas in Scotland. Schools were selected to represent a balance geographically as well as in terms of size, SIMD and a variety of curriculum models. In advance of these school visits, participating schools were provided with questionnaires for focus-group participants and also a link to the electronic survey with the following request: ‘…please also find below a link to our Survey Monkey questionnaire. It would be helpful to us if your teaching staff would take the time to participate in this survey within two weeks of our visit.
A short survey has been designed to capture the views of teaching staff on the extent to which the new National Qualifications have achieved their aims. Participants can provide this feedback by using the text boxes contained within the survey, or by sending comments directly to [email protected]. The survey can be accessed at: https://www.surveymonkey.com/r/NationalCourseDesignandAssessment’.
Data extraction Responses to the survey were downloaded as PDFs for overall responses and for responses for each subject (percentages of all respondents for each item, and all free text responses) and in Excel spreadsheets giving numbers of respondents and their responses for each item, for all respondents and for respondents for each subject. Results The total number of respondents to 7 January 2016, when the results presented below were extracted, was 337. Some respondents who began the survey did not rate any statements. Statements were rated by between 288 and 331 respondents (85%-98% of total respondents), representing 10-12% of all teachers that it was intended could have completed the survey (being a total of 2843 teaching staff in the 40 schools surveyed by 7 January 2016). Information on respondents/schools was not requested and so we do not know how the responses obtained were distributed across schools, and whether all schools surveyed are represented in the results. The numbers of respondents associated with subjects are detailed in Appendix 1: 582 selections were made, reflecting multiple selections by some respondents.
48
All subjects except the following were represented among survey respondents: Cantonese, Care, Childcare and Development, Mandarin, Music Technology, People and Society (N2), Practical Electronics, Sociology and Urdu. The numbers of respondents for each subject range from one (for seven different subjects) to 41 for English and 47 for Mathematics. The mean number of respondents for each subject option provided was eight and the median was four. Responses overall Graphical representations of the results of ratings provided by all respondents are presented in Appendix 2. Taking teachers of all subjects together, the ‘rating average’ was negative (tending towards disagreement) for all statements except the following, where the ‘rating average’ (tending towards agreement) was positive:
‘The new qualifications have an increased emphasis on skills.’ (Q2)
‘The new qualifications are inclusive. (This means that they have been designed to present no unnecessary barriers to learners on the grounds of socio-economic background or any of the protected characteristics of disability, race, age, religion or belief, sex, gender reassignment, pregnancy and maternity or sexual orientation.)’ (Q3)
‘The new qualifications have relevance in today’s world.’ (Q17)
‘Assessment for the new qualifications challenges learners.’(Q26)
However, all statements overall received a mixture of both positive and negative ratings, indicating an often wide range of views among respondents. The % of respondents selecting ‘neither agree nor disagree’ ranged from 8% (‘Q21. The overall approach to assessment reduces the time learners spend on assessment for certification and allows more time for learning’) to 40% (‘Q17. The new qualifications have relevance in today’s world), with a median of 22%. For 12 of the 25 statements, between 20% and 25% of respondents selected ‘Neither agree nor disagree’ which seems to indicate a significant degree of ambivalence in relation to almost half of the survey statements. Where respondents either ‘disagreed’ or ‘strongly disagreed’ the much more common response was ‘disagreed’. Similarly, where the response was ‘agreed’ or ‘strongly agreed’ the much more common response was ‘agreed’. The two statements with which the highest percentage of respondents either agreed or strongly agreed were:
‘Q3. The new qualifications are inclusive. (This means that they have been designed to present no unnecessary barriers to learners on the grounds of socio-economic background or any of the protected characteristics of disability, race, age, religion or belief, sex, gender reassignment, pregnancy and maternity or sexual orientation.)’ (49.24%)
‘Q2. The new qualifications have an increased emphasis on skills.’ (43.5%)
49
The two statements with which the highest percentage of respondents either disagreed or strongly disagreed were:
‘Q21. The overall approach to assessment reduces the time learners spend on assessment for certification and allows more time for learning.’ (87.7%)
‘Q25. Assessment for the new qualifications motivates learners.’ (73.24) Responses by subject Since many of the subjects had only one or a very small number of respondents, a selection of eight subjects was made to represent a range of subjects across different curriculum areas for which at least 14 respondents completed the survey. The eight subjects chosen are shown in Table 1. Note that many respondents indicated familiarity with a number of subjects, as shown in the table. Table 1: Subjects selected for inter-subject comparisons, and other subjects familiar
to those respondents.
Subject (total
respondents)
Experience of other Curriculum for Excellence subjects indicated by all
respondents for this subject (number of respondents for each)
Art and Design
(16)
Art and Design (Design) (5); Art and Design (Expressive) (5); Biology (1); Design
and Manufacture (1); English (1); Graphic Communication (1); History (1);
Lifeskills Mathematics (1); Music (1)
Chemistry (27) Biology (10); Environmental Science (2); Human Biology (1); Psychology (1)
Computing
Science (20)
Accounting (1); Admin and IT (4); Business (4); Business in Practice (N2) (1);
Business Management (2); Health and Food Technology (1); Hospitality
(Practical Cookery) (1); ICT (N2) (1); Physical Education (1)
English (41) All subjects (2); Admin and IT (2); Art and Design (1); Biology (1); ESOL (1);
History (1); Lifeskills Mathematics (1); Literacy and Numeracy (1); Mathematics
(1); Media (3)
French (21) Gaelic (Learners) (3); German (8); Gàidhlig (3); Italian (2); Spanish (10)
History (27) Admin and IT (1); Art and Design (1); Biology (1); Business (1); Business
Management (1); Classical Studies (1); English (1); German (1); Lifeskills
Mathematics (1); Literacy and Numeracy (1); Mathematics (1); Modern Studies
(8); Philosophy (2); Politics (1); RMPS (4); Social Subjects (N2)
Mathematics
(47)
All subjects (2); Art and Design (1); Biology (1); English (1); History (1); Lifeskills
Mathematics (20); Literacy and Numeracy (7); Mathematics of Mechanics (2);
RMPS (1); Statistics (2)
Physical
Education (17)
Accounting (1); Admin and IT (1); Business (1); Computing Science (1); Dance
(2); Health and Food Technology (2); Hospitality (Practical Cookery) (3)
Examples of the responses to statements by this selection of individual subjects are presented in Appendices 3 and 4, with graphs shown together for a single statement to allow comparisons between subjects to be made. For most items there were apparent differences between subjects (see Appendix 3 for one example).
50
In some cases the difference between subjects is striking. For example, 58% of History teachers agreed with the statement ‘The new qualifications provide clear and smooth progression between different levels (eg National 4 to National 5, National 5 to Higher)’ (Q10) and 33% of History teachers disagreed or strongly disagreed with it; whereas only 20% of Mathematics teachers agreed with the same statement and 71% disagreed or strongly disagreed. In addition, as is the case for responses overall, there is evidence of a range of views within subjects for most items. Some striking examples of this are:
57% of History teachers agreed (or strongly agreed) but 43% disagreed (or strongly disagreed) with the statement ‘The new qualifications support the values, purposes and principles of Curriculum for Excellence’ (Q19).
37% of Computing Science teachers agreed to some extent with the statement (Q22) ‘The overall approach to assessment has more focus on skills’, 32% disagreed with it and 32% neither agreed nor disagreed.
44% of Chemistry teachers agreed to some extent with the statement ‘Assessment for the new qualifications challenges learners’ (Q26), while 36% of them disagreed with it to some extent. Similarly, 45% of English teachers agreed to some extent with that statement, and 42% disagreed with it to some extent.
Positive and negative views were expressed within subjects for almost all statements. This was the case even when the responses were strongly negative overall, such as for the statement ‘The overall approach to assessment reduces the time learners spend on assessment for certification and allows more time for learning’ (Q21). Almost 88% of all respondents either disagreed or strongly disagreed with that statement and yet while 81% of Mathematics teachers disagreed or strongly disagreed with it (77% strongly disagreed), 9% agreed with the statement and a further 9% neither agreed nor disagreed. On the other hand, there is noticeable agreement between subjects for some statements (see Appendix 4 for one example). For example, all selected subjects overall largely disagreed with the statement ‘The new qualifications are trusted’ (Q16), and with the statement ‘The overall approach to assessment reduces the time learners spend on assessment for certification and allows more time for learning’ (Q21). Although the responses overall showed that some respondents agreed and some disagreed with each statement, within subjects some statements received no positive responses. Among the eight subjects whose results are presented in Appendices 3 and 4, no Physical Education teachers agreed with the statement ‘Pupils find the new qualifications easy to understand’; no teachers of Art and Design, Computing Science or English agreed that ‘Parents find the new qualifications easy to understand’; and no teachers of Chemistry, English, French and Physical Education agreed that ‘Employers find the new qualifications easy to understand’. No Physical Education teachers agreed that ‘The new qualifications provide clear and smooth progression on to post-school learning; and no teachers of Art and Design or Physical Education agreed that ‘The new qualifications provide clear and smooth progression on to post-school employment’. No English teachers agreed that ‘The new qualifications are trusted’. No Chemistry, History or Physical Education teachers agreed that ‘The overall approach to learning reduces the time learners spend on assessment and allows more time for learning’. For the eight subjects selected, some teachers agreed and some disagreed with all other statements.
51
Comparison of two subjects with most participants
As might be expected given the number of teachers of these subjects, the survey elicited
more respondents for English and Mathematics than for any other subject. English had 41
respondents and Mathematics had 47; no other subject provided more than 27 respondents.
An examination of responses within and between these two subjects reveals typical
similarities and differences within and between subjects, as shown in Table 2.
Table 2: Comparison of responses of English and Mathematics teachers to survey
items
SURVEY ITEM ENGLISH MATHEMATICS
≤ 30%
selected
‘Agree’ or
‘Strongly
agree’
≤ 20%
selected
‘Neither
agree nor
disagree’
≤ 30%
selected
‘Disagree’ or
‘Strongly
disagree’
≤ 30%
selected
‘Agree’ or
‘Strongly
agree’
≤ 20%
selected
‘Neither
agree nor
disagree’
≤ 30%
selected
‘Disagree’ or
‘Strongly
disagree’
Q2. The new
qualifications have an
increased emphasis on
skills.’
‘Q3. The new
qualifications are
inclusive.*
‘Q4. Each of the new
qualifications is
coherent in design and
content.’
‘Q5. Staff find the new
qualifications easy to
understand.’
‘Q6. Pupils find the new
qualifications easy to
understand.’
‘Q7. Parents find the
new qualifications easy
to understand.’
‘Q8. Employers find the
new qualifications easy
to understand.’
‘Q9. The new
qualifications provide a
seamless transition from
the Experiences and
Outcomes of the broad
general education.’
52
‘Q10. Provide clear and
smooth progression
between different levels
(eg National 4 to
National 5, National 5 to
Higher).’
‘Q11. The new
qualifications provide
clear and smooth
progression on to post-
school learning.’
‘Q12. The new
qualifications provide
clear and smooth
progression on to post-
school employment.’
‘Q13. The new
qualifications meet the
needs of learners of all
abilities in their
progression from prior
levels of achievement.’
‘Q14. The new
qualifications provide
opportunities for
learners to develop at
different times/rates in
different areas across
the curriculum.’
‘Q15. The new
qualifications maintain
high standards.’
‘Q16. The new
qualifications are
trusted.’
‘Q17. The new
qualifications have
relevance in today’s
world.’
‘Q18. The new
qualifications support
the learning of the new
curriculum, including its
breadth.’
53
‘Q19. The new
qualifications support
the values, purposes
and principles of
Curriculum for
Excellence.’
Q21. The overall
approach to
assessment reduces the
time learners spend on
assessment for
certification and allows
more time for learning.’
‘Q22. The overall
approach to
assessment has more
focus on skills.’
‘Q23. The overall
approach to
assessment has more
focus on integration with
other aspects of
learning.’
‘Q24. Assessment for
the new qualifications
supports learners.’
‘Q25. Assessment for
the new qualifications
motivates learners.’
‘Q26. Assessment for
the new qualifications
challenges learners.’
‘Q27. Assessment for
the new qualifications
has more scope for
personalisation and
choice.’
*For clarity, the term ‘inclusive’ was defined within the survey item as follows: ‘(This means
that they have been designed to present no unnecessary barriers to learners on the grounds
of socio-economic background or any of the protected characteristics of disability, race, age,
religion or belief, sex, gender reassignment, pregnancy and maternity or sexual orientation.)’
Discussion Survey results should be considered in the light of a number of factors which might
reasonably have affected the results. Some of these are considered below.
The instructions accompanying the link to the electronic survey regarding its distribution may
have been misinterpreted by some schools, such that the survey was not made available to
54
all teaching staff in the school as had been intended (and previously advised), but was only
provided to teaching staff participating in the focus groups. In these circumstances, not all
teaching staff in all schools will have been given the opportunity to complete the survey. In
addition, among those who did have the opportunity to complete the survey, respondents
were self-selecting. Therefore, survey respondents may comprise a sample that is biased to
some extent rather than a sample that is representative of the entire teacher population or
even of the teacher population in the selected schools.
In addition, many of the schools to which the survey was sent had in the preceding year
(2014-15) offered the old Higher in some or all subjects, and this is likely to have had an
impact on responses to some of the survey items, for example those relating to progression
between levels, and coherence.
Finally, the survey coincided with an EIS campaign/ballot focusing on ‘…excessive workload;
over-assessment around Units; the lack of fallback from Higher to National 5; the perceived
value of National 4; senior phase articulation with broad general education; elaborate and
overly bureaucratic verification procedures; and timetabling pressures’. It is therefore
possible that survey responses were influenced to some extent by that campaign.
Although the response rate for an electronic survey is reasonable, the total numbers of
teachers responding for each subject are relatively low, other than for Mathematics and
English, and so caution should be exercised in reaching conclusions at a subject level for
most subjects. Furthermore, overall responses will have been more influenced by subjects
with larger numbers of respondents. The subjects with the highest numbers of respondents
(ranging from 17-47) were Biology, Chemistry, Computing Science, English, French, History,
Lifeskills Mathematics, Mathematics, Modern Studies, Physical Education, and Science.
Respondents were able to select more than one subject with which they were familiar prior to
rating their agreement or disagreement with each statement. While this will have had no
impact on overall responses (ratings for all subjects taken together), where respondents
selected more than one subject (see Table 1) their responses will be reflected in the results
for each of those subjects. The responses the analysis has attributed to one subject may
therefore have been informed to some extent by experience with other subjects, which may
or may not have been within the same curriculum area.
The fairly high level of ambivalence shown in responses to many of the survey items may
reflect the period during which the survey was conducted — a relatively early stage in a
significant change process — and/or the inexperience of some schools at that time with the
new Higher. It would be interesting to repeat the survey in subsequent years, when it might
be expected that respondents would be increasingly inclined to express either a positive or
negative view.
Seventy-two percent of all respondents disagreed or strongly disagreed with the statement
‘Staff find the new qualifications easy to understand’ (Q5), and a high level of disagreement
with this statement was also reflected at subject level. The finding, overall and at a subject
level, that respondents believe the new qualifications are difficult to understand is
concerning, but may be to some extent a reflection of the survey being conducted at an early
stage during implementation of a very new system of qualifications. Any current lack of
understanding seems likely to have had an influence on responses to many other items.
55
The statements relating to how easy the qualifications are to understand (Q5-Q8) are unlikely
to be independent: where a respondent believes that staff do not find the new qualifications
are easy to understand it is highly unlikely that he/she will find that other groups (pupils,
parents and employers) will find the new qualifications easy to understand.
The free text responses to Q20 and Q28 have been analysed separately to identify broad
themes reflected in comments by teachers participating in the survey. It is likely that some
respondents will have provided the same views in both focus group and survey, and so these
data may therefore not be independent. However, the responses provided to Q20 and Q28
were not constrained by focus group questions. They may therefore reflect more clearly than
the focus group responses the most important or most immediate issues concerning this
sample of teachers, thereby perhaps providing a slightly different perspective from that of the
focus group findings.
The new system of qualifications is part of a transformational change programme for
Scotland’s curriculum, and is the culmination of the largest programme of qualifications
development ever undertaken in Scotland. Given the unprecedented demands of such a
change programme upon centre staff at the time the survey was conducted, the level of
agreement with the survey statements — representing SQA’s ambitions for the new system
of qualifications — while low overall is perhaps higher than might have been expected. It
does, however, usefully identify some areas and perhaps some subjects where SQA’s
ambitions are at present far from being realised, and may help to guide appropriate actions
by relevant stakeholders.
56
APPENDIX 1 Numbers of respondents selecting each subject option in response to Q1 which was ‘Your
experience of the new qualifications may relate to all Curriculum for Excellence subjects or
only some. Please choose from the list below and consider only those subjects when
responding to the survey questions that follow. My experience of CfE relates to …’
SUBJECT Numbers of respondents
All subjects 14
Accounting 7
Administration and IT 12
Art and Design 16
Art and Design (Design) 5
Art and Design (Expressive) 5
Biology 27
Business 10
Business in Practice (National 2) 2
Business Management 9
Cantonese 0
Care 0
Chemistry 27
Childcare and Development 0
Classical Studies 5
Computing Science 20
Creative Arts 1
Dance 2
Design and Manufacture 8
Design and Technology 5
Drama 11
Economics 2
Engineering Science 2
English 41
English and Communication (National 2) 2
Environmental Science 3
ESOL 3
Fashion and Textile Technology 1
Food, Health and Wellbeing (National 2) 1
French 21
Gaelic (Learners) 3
Geography 15
German 10
Graphic Communication 10
Gàidhlig 3
Health and Food Technology 5
History 27
Hospitality: Practical Cake Craft 2
Hospitality: Practical Cookery 13
Human Biology 2
Information and Communications Technology (National 2) 2
Italian 3
Latin 4
Lifeskills Mathematics 22
Literacy and Numeracy 11
Mandarin 0
57
Mathematics 47
Mathematics of Mechanics 2
Media 5
Modern Studies 17
Music 13
Music Technology 0
National 1 and 2 1
People and Society (National 2) 0
Performance Arts (National 2) 2
Philosophy 5
Photography 1
Physical Education 17
Physics 16
Politics 1
Practical Craft Skills 2
Practical Electronics 0
Practical Metalworking 4
Practical Woodworking 6
Psychology 1
Religious, Moral and Philosophical Studies 10
Science 20
Science in the Environment (National 2) 1
Social Subjects (National 2) 3
Sociology 0
Spanish 12
Statistics 2
Urdu 0