National Course design and assessment

SQA fieldwork visits — engagement and focus group discussions with centres delivering new National Qualifications

This edition: May 2016, version 1.0

© Scottish Qualifications Authority

1 Background 1

1.1 Acknowledgements 1

1.2 Rationale 1

1.3 Research methodology 1

1.4 Impact of curriculum structures and option choices 3

2 Executive summary of themes and issues emerging 3

2.1 Broad general education 4

2.2 Assessment 4

2.3 Personalisation and choice 5

2.4 Documentation 5

2.5 Developing the Young Workforce 6

2.6 Further support 6

3 Conclusions 7

4 Annexes 8

4.1 Questions for senior management team 8

4.2 Questions for teaching staff 10

4.3 Questions for candidates 12

4.4 Background questionnaire for teaching staff 13

4.5 Background questionnaire for candidates 15

4.6 Summary of senior management team focus group responses 17

4.7 Summary of teacher focus group responses 29

4.8 Summary of candidate focus group responses 35

4.9 Statistical data — teachers (background questionnaires) 43

4.10 Statistical data — candidates (background questionnaires) 44

4.11 Online survey analysis and commentary 46

1

1 Background

1.1 Acknowledgements Thanks to the Curriculum for Excellence (CfE) support team for their help in organising the

focus group visits, and in developing the focus group materials and questionnaires which the

teams used as the basis for their work in centres, and to the team leaders and scribes who

coordinated the focus group work.

Thanks should also go to colleagues in the Research, Policy, Statistics and Standards team

(RPSS) for helping to develop the questions used in the focus group work, and to the

Customer Service Managers team who helped in coordinating the school centre visits.

Above all, thanks to the senior managers in SQA centres for giving up their time to meet with

the teams and helping to organise the meetings with the focus groups. Also, many thanks to

the teachers and candidates for taking the time to meet with the teams, and complete the

detailed questionnaires.

1.2 Rationale This series of engagements with centres/colleges/local education authorities (LEAs) and

other partners is part of a suite of SQA activities designed to evaluate the first phases of the

new National Qualifications (NQs). Its purpose was to attempt to evaluate at first hand:

 the impact that implementation of the new NQs has had on centres in the period 2012-16

 what, if any, effects this implementation continues to have

Whilst not linked directly to the thematic review conducted by colleagues in RPSS, some of

the observations and data included here do have links to some of the data included in that

review, in terms of opinions gathered from National Qualifications Support Teams (NQSTs)

and an alignment of questions asked at interview.

1.3 Research methodology The Qualifications Development (CfE) team formed teams of three in cooperation with SQA’s

Customer Support Manager (CSM) team to visit centres in each local authority. A minimum

of one centre per authority was the target for this exercise, although the number was

extended in larger authorities. The centres were chosen to reflect the wide variety of contexts

with which centres operate, so the centres included in the research were a spectrum of rural

and urban; large and small by roll; denominational and non-denominational; reflective of free

meal entitlement provision. This resulted in visits being made to 40 centres (including the

results from two pilot schools) during November and December 2015.

It was also proposed that the teams have a structured conversation with a number of college

providers, parents, and local authority representatives to reflect their perspectives on the new

2

NQs. This process is still ongoing — meetings have not yet taken place with college staff and

other recent meetings are still being written up.

Each team visiting a centre consisted of a team leader (one of four in this role) who acted as

moderator, and a scribe from the Qualifications Development (CfE) team. This was to ensure

validity and reliability in the process of recording the focus group responses. In addition, the

CSM member acted in an introductory and liaison capacity and as an additional note-taker

where necessary. The questions for the three focus groups were developed by the

Qualifications Development (CfE) team in liaison with RPSS. The questions were sent to the

centres to allow the individuals and groups chosen to discuss and prepare any responses in

advance (and to compensate for limited time in some cases), with the proviso that the

moderators might prompt or probe deeper in conversations in individual cases where it was

felt appropriate.

The Qualifications Development (CfE) team asked for the groups to be formed in the

following way where possible, taking account of local circumstances:

 8-10 pupils from S5-S6, ie those candidates who had a fuller experience of the new NQs

 6-8 classroom teachers/principal teachers/faculty heads ie to reflect the balance of

teaching staff and the variety of departmental organisation across the country

 as many of the senior management team as could be available on the day for the

discussions at school management level

The timing of the focus groups was agreed in advance with the centres to fit in with their own

timetabling structures. This meant the group’s discussion could vary from 40 minutes to one

hour. The pupil and teacher groups were also asked to complete a background questionnaire

(see Annexes 4 and 5) to provide information about certain aspects of their experience of the

new NQs, and any specific issues that had emerged for them over the implementation. This

was designed to add experiential depth to the data being gathered in the focus groups and

help to develop a picture across curriculum areas and subjects.

The design of the questions was intended to elicit and identify generic issues across centres

and areas of the country. Some of the questions posed were common to all groups to elicit

comparative responses. However, given the make-up of at least two of the groups

(candidates and teachers), it was accepted that some of the opinions expressed would be

derived, both in the questionnaire and focus groups, from specific subject-based

experiences. That may be reflected below, wherever appropriate, to illustrate any elaborated

responses. The design of the questions incorporated, in some cases, a specific set of

potential responses to help the scribes record the information, but these were not intended to

deflect the conversation from more elaborated responses where appropriate. Broad scope

was given for the occasionally complex recording of these.

The questions were piloted in two centres to help the Qualifications Development (CfE) team

to identify any initial problems, either with the questions or the timing of the focus groups, but

by and large the methodology chosen seemed to be appropriate to the task. In addition,

centres were invited to complete a set of online survey questions, in order to give greater

depth and detail to the data being gathered. The core of the online survey analysis and data

is included in Annexe 11.

3

Great care was taken to reassure centres that their contributions would be anonymised and

no references would be made which could identify where specific information had originated.

Accordingly, there are no school-specific references in this document.

1.4 Impact of curriculum structures and option choices It should be noted that the spread of centres included in the research presented a variety of

curriculum models and subject choices for candidates, and the candidate questionnaire

responses reflected this. Generally, centres stipulated that they were using a 3+3 model,

although it was apparent that, in practical terms, some centres still had candidates making

some form of option choice at the end of S2. In addition, it was clear that there was a

spectrum of arrangements at local authority level. Initially, it seems that most of the LEAs

had given centres a fairly free rein in developing curriculum models on a local basis.

However, it was also apparent in discussions that some LEAs were reviewing these

arrangements in order to design curriculum models and subject choices across the LEA.

Whether these changes were being made for strategic or cost reasons was not always clear

to the focus group teams.

In looking at the candidate questionnaires, it is clear that the subject choices being made for

the senior phase starting in S4 still reflect a high number of subjects being taken at that stage

— up to nine in some cases, reducing to five in S5 — with a variety of Higher and Advanced

Higher in S6. Some centres have subsequently