Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | christian-nichols |
View: | 213 times |
Download: | 0 times |
The National Student Survey (NSS)
Penny Jones, Strategic Planning Office
Tracy Goslar and Miles Willey , Academic Standards & Partnership
Wednesday 16 March 2011 (Moulsecoomb)Friday 18 March 2011 (Eastbourne)
• NSS began in 2005 and is a key element in the Quality Assurance Framework
• The QAF, which also includes institutional audit, is the means by which HEFCE meets its legal obligations for ensuring the quality of publicly funded higher education.
• NSS was and remains a key element in the commitment to provide public information about quality and standards of publicly funded higher education (www.unistats.com)
The National Student Survey (NSS)
• NSS is becoming increasingly integrated into the national quality assurance system, as evidenced by plans to further develop and enhance the survey
• Student feedback from the NSS is considered alongside other student evaluation as a central aspect of the University’s annual Academic Health process
• Nationally, the student experience is viewed as a central pillar of quality assurance systems
Integration of the NSS into quality assurance systems
• 22 questions • 5 point scale – Definitely agree to definitely disagree • 6 main sections, Questions 1-21
1. The teaching on my course
2. Assessment and feedback
3. Academic support
4. Organisation and Management
5. Learning resources
6. Personal Development • Question 22 – overall satisfaction • Additional questions • Comments
•
The survey
• The survey is anonymous
• To receive published results for the NSS, the University must receive a response rate of at least 50%
• Results at course level are available for internal use only where 10 or more respondents complete the survey
• Responses below the threshold of 10 are published at the JACS subject level on the Unistats website
Publication of response rates
• Samples not representative
• Comparisons or ranking of whole institutions in league tables
institutions fall within narrow range covered by sampling error
Subject mix in different institutions
• Students expectations different
• Opinion not truth
• Easy to manipulate
• UNISTATS
Why we don’t like the NSS - survey criticisms
• Free market research! Important tool to improve student experience and highlight broad areas where there are issues, rather than ranking tool
• Use free text comments to contextualise figures
• Work in conjunction with SU and course reps to check what is real
• Use alongside internal evaluation - modules etc.
• Benchmarking against sector
• Issues discussed in report to HEFCE ‘Enhancing and Developing the NSS’ http://www.hefce.ac.uk/pubs/rdreports/2010/rd12_10/
Why we do like the NSS...
• Staffcentral link: http://staffcentral.brighton.ac.uk/nss
• NSS website: www.thestudentsurvey.com
• General enquiries: Tracy Goslar (Academic Standards & Partnership) – [email protected]
• Analysis of response rates and results: Penny Jones (Strategic Planning Office) – ext 2052
• Marketing materials: Susan Lightfoot (Marketing & Communications) – ext 2221
Useful contacts
• Response Rates 2011
• Trends and demographics 2006 to 2010
• Further Analysis
Numbers!
2010 First time UoB response rate higher
than sector!
Response Rates
50%
52%
54%
56%58%
60%
62%
64%
66%
68%
2006 2007 2008 2009 2010
University of Brighton % Response Rate Versus the Sector 2006 to 2010
UoB %
Sector %
50%
55%
60%
65%
70%
0
500
1000
1500
2000
2500
3000
2006 2007 2008 2009 2010
University of Brighton % Response Rate and Actual Number of Respondents
UoB No
UoB %
74% more respondents than 2006
Response Rates 2011 – versus previous years
Course Name: Course Code: School Delivering:
H0051 Brighton (University of) H0051 Brighton (University of) H0051 Brighton (University of)
Sample Eligible
2011
3907
Sample Eligible
2010
4022
Sample Eligible
2009
424845.2
65.6
57.1
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
1M 1F 2W 3M 3F 4W 5M 5F 6W 7M 7F 8W 9M 9F 10W 11M 11F 12W 13M 13F 14W 15M 15F 16W 17M 17F 18W
% R
espo
nse
Rate
Week Number & Day
2011
2010
2009
Response Rates 2011 – versus the sector
45.2
45.3
0
5
10
15
20
25
30
35
40
45
50
1M 1W 1F 2M 2W 2F 3M 3W 3F 4M 4W 4F 5M 5W 5F 6M 6W 6F 7M 7W 7F 8M 8W 8F 9M 9W 9F
10M
10W 10
F
11M
11W 11
F
12M
12W 12
F
13M
13W 13
F
14M
14W 14
F
15M
15W 15
F
16M
16W 16
F
17M
17W 17
F
18M
18W 18
F
% R
espo
nse
Rate
Week & Day
University of Brighton National Student Survey % Response Rate Versus the Sector
UoB inc NHS 2011
Sector inc NHS 2011
• Trends
Mode Gender Domicile Age Ethnicity Level Disability NHS
• Look at the optional set of questions
Presentation of Trends
National Student Survey Findings and Trends 2006 to 2009 (HEFCE 2010/18)
Mode – Full time and Part Time
2006 2007 2008 2009 2010
Brighton Full Time 78 77 80 79 78
Brighton Part Time 80 83 88 89 86
Sector Full Time 80 81 82 81 81
Sector Part Time 86 86 85 84 87
70
72
74
76
78
80
82
84
86
88
90
% Agree Overall Satisfaction in Full time and Part time Students at UoB and in the Sector
Brighton Full Time
Brighton Part Time
Sector Full Time
Sector Part Time
‘...the profile of satisfaction scores for those on part-time courses was significantly different to that of those studying on full-time courses. The question categories with the largest differences were Assessment and feedback, where part-time students were more satisfied, and Learning resources and Personal development, where part-time students were less satisfied.’ (HEFCE 2010)
Gender
2006 2007 2008 2009 2010
Brighton Female 78 76 80 81 78
Brighton Male 77 79 81 79 80
Sector Female 81 82 82 82 82
Sector Male 80 81 82 81 82
75
76
77
78
79
80
81
82
83
% Agree Overall Satisfaction, by Year, by Gender, at the University of Brighton and in the Sector
Brighton Female
Brighton Male
Sector Female
Sector Male
‘While Overall satisfaction of female and male students was similar to the global score significant differences were observed in other categories, most notably for Academic support and Learning resources.’ (HEFCE 2010)
Domicile
0
10
20
30
40
50
60
70
80
90
100
2006 2007 2008 2009 2010
% Agree Overall Satisfaction by Domicile at University of Brighton
EU (excl UK)
Other
UK
0
10
20
30
40
50
60
70
80
90
100
2006 2007 2008 2009 2010
% Agree Overall Satisfaction by Domicile in the Sector
EU (excl UK)
Other
UK
‘Students from the EU and international students were significantly less satisfied with Teaching and learning than students from the UK. For Assessment and feedback, students from the EU were significantly less satisfied and international students were significantly more satisfied than the global score.’(HEFCE 2010)
Age
70
72
74
76
78
80
82
84
2006 2007 2008 2009 2010
% Agree Overall Satisfaction by Age, at Brighton and in the Sector.
UoB Mature
UoB Young
Sector Mature
Sector Young
‘...students in the age groups 21-24 and over 25 were significantly less satisfied than students aged under 21, in the questions on Academic support, Organisation and management, Learning resources and Overall satisfaction. However, students aged over 25 were significantly more satisfied than the global score for the category Assessment and feedback.’ (HEFCE 2010)
Ethnicity
50
55
60
65
70
75
80
85
90
2006 2007 2008 2009 2010
% Agree Overall Satisfaction, by Year, and Ethnicity, at University of Brighton.
Asian
Black
Not Known
Other
White
50
55
60
65
70
75
80
85
90
2006 2007 2008 2009 2010
% Agree Overall Satisfaction, by Year, and Ethnicity, in the Sector.
Asian
Black
Not Known
Other
White
University of BrightonEthnicity 2006 2007 2008 2009 2010Asian 99 104 53 106 122Black 55 59 24 78 94Not Known 87 107 124 217 247Other 55 65 49 95 127White 1194 1418 1033 1792 2009
‘Satisfaction profiles varied significantly depending on students’ ethnic background. However, the only question category where all the differences in satisfaction score were significant was Teaching and learning; where students from a White ethnic background were more satisfied than the global score and students from all minority ethnic backgrounds were significantly less satisfied than the global score. Similar trends were seen in the question categories Academic support, Learning resources and Overall satisfaction. However, the results were not all of statistical significance, and the analysis does not take into account additional factors such as subject area or institution which may also affect scores Minority ethnic groups in this context are defined to be students from Asian, Black, Chinese, Mixed or Other ethnic backgrounds.
Level of Study
65
67
69
71
73
75
77
79
81
83
85
2006 2007 2008 2009 2010
% Agree Overall Satisfaction by Year, by Level, at UoB and the Sector
UoB First Degree
UoB Other Undergraduate
Sector First Degree
Sector Other Undergraduate
‘The largest differences from the global score for respondents studying for foundation degrees and other qualifications are for question categories Assessment and feedback, where they are more satisfied than first degree students, and Organisation and management, where they are less satisfied.’ (HEFCE 2010)
Disability
60
65
70
75
80
85
2006 2007 2008 2009 2010
% Agree Overall Satisfaction, by Year and by Disability Status at University of Brighton.
UoB Disability (excl Dyslexia)
UoB Dyslexia
UoB No known Disability
60
65
70
75
80
85
2006 2007 2008 2009 2010
% Agree Overall Satisfaction, by Year and by Disability Status in the Sector.
Sector Disability (excl Dyslexia)
Sector Dyslexia
Sector No known Disability
‘Disabled students were significantly less satisfied than the global scores in five of the seven question categories (see Figure 12). The question categories with the lowest satisfaction scores were Organisation and management and Overall satisfaction. ‘ (HEFCE 2010)
NHS Funded Students
60
65
70
75
80
85
2006 2007 2008 2009 2010
% Agree Overall Satisfaction, by Year, by NHS, at Brighton and the Sector
UoB NHS student
UoB Non-NHS student
Sector NHS student
Sector Non-NHS student
‘the profile of satisfaction scores for NHS-funded students was significantly different to that of non-NHS funded students. The question categories with the largest differences were Organisation and management, where NHS-funded students were less satisfied, and Personal development, where NHS-funded students were more satisfied.’ (HEFCE 2010)
Optional Questions
40
45
50
55
60
65
70
75
80
85
90
2007 2008 2009 2010
% Agree satisfied with Work Placements, by Year, UoB compared to the Sector
Brighton (University of)
England
HEI
Top Quartile
40
45
50
55
60
65
70
75
80
85
90
2007 2008 2009 2010
% Agree satisfied with Feedback from Students, by Year, UoB compared to the Sector
Brighton (University of)
England
HEI
Top Quartile
40
45
50
55
60
65
70
75
80
85
90
2007 2008 2009 2010
% Agree satisfied with Intellectual Motivation, by Year, UoB compared to the Sector
Brighton (University of)
England
HEI
Top Quartile
40
45
50
55
60
65
70
75
80
85
90
2007 2008 2009 2010
% Agree satisfied with Social Opportunities, by Year, UoB compared to the Sector
Brighton (University of)
England
HEI
Top Quartile
• Turn the presentation into a more detailed paper – or wait to next year and do a copy of the HEFCE
• Satisfaction by Campus
• Partner Colleges – report available from SPO
Further Analysis
HEFCE Publication - National Student Survey Findings and Trends 2006 to 2009. http://www.hefce.ac.uk/pubs/hefce/2010/10_18/
http://www.heacademy.ac.uk/ourwork/supportingresearch/nss
Literature
Comparative review of national surveys of undergraduate students
Degree Attainment, ethnicity and gender
Dimensions of Quality
Enhancing and Developing the National Student Survey
EvidenceNet Wiki pages on the NSS
Exploratory evaluation of the use of the National Student Survey Results Dissemination Website
Exploring the NSS: Assessment and Feedback Issues
Interpreting National Student Survey (NSS) data using multi-level modelling
Multilevel analysis of discipline differences - Herb Marsh (August 2008)
National Student Survey (NSS) institutional case studies
National Student Survey: Findings and trends 2006 to 2009
NSS 2007 optional items - Institution Guide and Statistical Report
NSS Resources on EvidenceNet
Paula Surridge's report: NSS Three years on
Why we shouldn’t use student surveys of teaching as satisfaction ratings