Deakin University CRICOS Provider Code: 00113B

PRACTICALLY AND PRODUCTIVELY ANALYSING COURSE EXPERIENCE QUESTIONNAIRE STUDENT COMMENT DATA

Stuart Palmer and Malcolm CampbellFaculty of Science, Engineering and Built Environment

Deakin University

Deakin University CRICOS Provider Code: 00113B

Survey text data is often under-utilised

Large-scale investigation of CEQ comments

160k+ CEQ comments from 2001-2004

Common themes identified

CEQuery software developed to automate the analysis of CEQ comment data

Five factor structure proposed…

Accessing the Student Voice - Scott

Deakin University CRICOS Provider Code: 00113B

CEQuery domains and sub-domains

1. Outcomes 2. Staff 3. Course design 4. Assessment 5. Support

Intellectual Teaching skills Practical-theory links Relevance Library

Work application/ career

Accessibility and responsiveness

Methods of learning & teaching Marking Learning resources

Further learning Quality and attitude Flexibility/responsiveness Expectations Infrastructure/ environment

Personal Practical experience (current)

Relevance (to work/life/discipline) Feedback/return Student

administration

Interpersonal Structure and expectations Standards Student services

Knowledge/skills Social affinity/support

Deakin University CRICOS Provider Code: 00113B

1. To validate the 2012 annual Faculty CEQ comment data set against the five domain model proposed previously by Scott (2006)

2. Identify how to best use this data, in practice, at the individual-Program-level as part of the wider use of SET data for the improvement of teaching and learning

Project aims

Deakin University CRICOS Provider Code: 00113B

Ranked overall comment hit counts by Faculty Program

0

20

40

60

80

100

120

140

160

Ove

rall

com

men

t cou

nt

Ranked order (by overall comment count) of all 55 Faculty programs

Needs Improvement Best Aspects

Deakin University CRICOS Provider Code: 00113B

Overall BA and NI comment hit count proportions

47.3%

21.8%

14.5%11.9%

2.6% 1.4% 0.4%

39.1%

26.3%

18.1%12.6%

4.0%

0%

10%

20%

30%

40%

50%

Course design Staff Outcomes Support Assessment Other Everything

This investigation As found by Scott 2006

40.8%

24.9%

14.4%8.9%

6.0% 3.5% 1.5%

43.2%

24.3%

16.5%9.5%

6.5%

0%

10%

20%

30%

40%

50%

Course design Staff Support Assessment Outcomes None Other

This investigation As found by Scott 2006

Deakin University CRICOS Provider Code: 00113B

Overall BA comment hit count proportions by School and Gender

0%

10%

20%

30%

40%

50%

Course design Staff Outcomes Support Assessment Other Everything

School A School B School C School D

0%

10%

20%

30%

40%

50%

Course design Staff Outcomes Support Assessment Other Everything

Female Male

Deakin University CRICOS Provider Code: 00113B

Where separately reported Programs have a significant degree of commonality, the comment data for those programs could be consolidated to form a larger pool

Where the number of CEQ responses is low, data from more than one year can be pooled to form a more useful data set

When Program comments are few…

Deakin University CRICOS Provider Code: 00113B

At the School- or Program-level, it is feasible to simply pass on the ‘raw’ CEQ comments directly to the relevant program leadership for consideration…

…in conjunction with other SET data, surveys of graduates and employers, benchmarking, advice from accreditation bodies, student focus groups, etc.

When Program comments are few…

Deakin University CRICOS Provider Code: 00113B

…support for Scott’s 5 factor structure

…manual coding of comments against the 5 factor model is feasible, and achieves a superior coding rate compared to CEQuery

…large number of Course Design comments, both BA and NI, related to practical aspects of a Program

Conclusions – We found…

Deakin University CRICOS Provider Code: 00113B

ow.ly/qlhzo

Thank you for your time