Bond University: eTEVAL Report User Guide For Teaching staff and administrators

Table of Contents eTEVAL Reports ....................................................................................................................................... 3

Accessing your eTEVAL reports ........................................................................................................... 3 LƴǘŜNJƴŜǘ 9ȄLJƭƻNJŜNJ π {ŜŎdzNJƛǘȅ {ŜǘǘƛƴƎǎ .................................................................................................. р

Content Analysis Tool ....................................................................................................................... 1п

!ŘƳƛƴƛǎǘNJŀǘƻNJ wŜLJƻNJǘ !ŎŎŜǎǎ ..ΦΦΦ.ΦΦΦΦΦΦΦΦ............................................................................................... т

eTEVAL Reports eTEVAL reports will be accessible from “O” week each semester ( after student results are released). Each Faculty educator/administrator will receive an email with an authentication link to access the EvaluationKIT Report dashboard. Please note: Access to the reports may not work on all mobile devices. EvaluationKIT is currently working on mobile device compatibility. This document is a step-by-step guide on how to access your eTEVAL results for educators/administrators. This guide is applicable to both the Subject and Educator reports. In addition to this document, Vishen Naidu: vnaidu@bond.edu.au, EXT:53393 is available for face-to-face and telephone supports regarding eTEVAL and report access and administration.

Levels of Access

• Educators will receive access to the relevant classes/subjects they taught during an evaluation period

• The Head of School (HoS) will receive access to all subject results at the Faculty School level • All Associate Deans – Teaching and Learning (AD-TL) will receive access to all results at the

Faculty level • The Deans will receive full access to their Faculty level results

Accessing your eTEVAL reports

Instructor report access

1. Click and follow the login link sent to your Bond staff email

2. The link will direct you to the EvaluationKIT dashboard. From here you will be able to view the reports for the Subject and Educator class results. Select the Survey results you wish to view.

3. The Course Results dashboard shows the list of subjects/classes that you taught during the semester. Select one of the following file formats from the dashboard: (all three files produce the same data in different formats) PDF report – provides a breakdown of each question using common descriptive statistics (Mean and Standard Deviations). This report will also provide a summary of the response rates for each question PDF+TEXT report – this report has a similar output as the one above and also includes comments (write-in responses) in no particular order. Excel spread sheet – shows the raw data responses in Excel

Internet Explorer - Security Settings

Automatically download files

In some cases your Internet Explorer browser will automatically block files from downloading as an added security measure in the newer versions.

To enable automatic prompting for file downloads follow the steps below:

Step 1:

In your Internet Explorer browser click on the “Tools” menu

Step 2:

Then click “Internet Options”

Step 3:

Click on the “Security” tab at the top and click “Custom level”

Step 4:

Scroll down till you see the heading “Downloads” under “Automatic prompting for file downloads” click “Enable” and then “OK”

Administrator report access Administrator level report access will also be available from “O” week. Selected Faculty members (HOS, AD-TL and the Deans) will gain full access to their Faculty/School level reports. Administrator access allows the user to filter and search all subjects/classes and the individual lecturers/tutors that taught during the semester. To access the overall Faculty/School reports:

1. Click and follow the login link sent to your Bond staff email

2. The link will direct you to the EvaluationKIT dashboard. From here you will be able to view your Faculty/School level reports for the Subject and Educator results. Select the Survey results you wish to view.

F

3. The next screen will prompt you to select a level. Depending on what level of access you have. You will need to select the hierarchy level to filter the results. For example, if you are the Head of School, you will only have access to your School level results.

4. Select one of the following file formats from the dashboard to view the aggregated results for that specific level: (all three files produce the same data in different formats) PDF report – provides a breakdown of each question using common descriptive statistics (Mean and Standard Deviations). This report will also provide a summary of the response rates for each question PDF+TEXT report – this report has a similar output as the one above and also includes comments (write-in responses) in no particular order. Excel spread sheet – shows the raw data responses in Excel

5. You can further refine your results by filtering down to the subject level by selecting the relevant program level under the Course-Level Results section and click Find Course

6. Select one of the relevant file formats for the individual subjects or select/tick one or more subjects and click Generate Batch Reports to download the selected results in PDF format

To search reports for individual lecturers/tutor: There are a couple of different ways in which you can access the results for the individual lecturer/tutor results. Follow steps 1 & 2 above, and select the Educator Survey report results.

1. Selecting the relevant program from the course level results list, click find course and find the subject/class, lecturer/tutor you require.

2. To further refine your search results, click “More”

For more comprehensive reports and results, from the results tab, select one of the following reports: Instructor Means Reporting – Using the filters and drop down boxes, this report can be customized to provide a breakdown of the individual educators’ performance. Benchmarks can also be added. TEVAL Summary Reporting – Using the filtering metrics, this report can also be fully customized to provide a comprehensive view of all educators’ results.

1. Hover your mouse over the Results tab and select one of the reports

Instructor Means Reporting:

TEVAL Summary Reporting

Understanding and Interpreting eTEVAL Reports

At Bond we administer two separate evaluations which use a five point Likert scale including a Not Applicable response. The Subject evaluation contains 5 questions and one open text comment field, and seeks to evaluate the quality of the subject and content. The Educator evaluation on the other hand contains 10 questions and two open text comment fields, and seeks to evaluate the quality of the teaching and delivery. It was important that this distinction was made between the two evaluations, as we wanted the students to understand that the way the subject is designed and structured is not always the responsibility of the educator delivering the lectures or tutorials.

Reports are made available to Subject convenors and Instructors in ‘O’ week of the following semester. The individual instructors only receive access to the subjects they taught in the previous semester, while the subject convenor receives access to all reports under the relevant subject. Within each faculty there are also set hierarchies of administrators that gain greater access to various Faculty level reports.

The following explanations will help identify the various sections within the reports.

Quantitative Data

a. Response Option – based on 5-point Likert scale including a Not Applicable response to gauge the level agreement or disagreement to the corresponding question.

b. Weight – typically Likert scale responses are given a score or weight to summarize the data, usually from 1-5 with five being the highest.

c. Frequency – refers to the number of times a response was received for the corresponding option

d. Percentage – is a percentage calculation of the response option

Note: The 3 digit suffix at the end of the title indicates Year [2 digits] and semester [1 digit] of study i.e. 122 – 2012, Semester 2

a.

b. c. d.

e.

f. g.

h.

e. Percent Responses – a graphical representation of the data in the previous column f. Return Rate – is the number of students who actually responded to the evaluation out of

the total number of student enrolments for the subject. A disparity occurs when the remaining students choose to opt-out/skip an evaluation.

g. MEAN - is the sum of the responses divided by the total number of responses for a given question. If we take the above (Q1) as an example, the MEAN of 4.15 suggests that the majority of the responses fell between Agree and Strongly Agree. STD (Standard Deviation) - is a measure of the spread of scores around the mean. If the STD is small, for example .01, it would imply that that the majority of the responses were not significantly different from the mean. If the STD is large, it would imply that responses varied significantly from the mean. In the above example, the STD of 0.84 infers there isn’t a significant variation from the MEAN and most of the responses actually fell within the Agree/Strongly Agree range.

h. Means – a graphical representation of the MEAN for the corresponding question

Content Analysis Tool (Qualitative Data)

In some cases where there are a significant number of comments or write-in responses, the reports will provide a breakdown/analysis of the common reoccurring themes among the comments received. The comments are further analysed using a comprehensive database that identifies the sentiment of the comments as being Positive, Neutral or Negative. These results are weighted against the number of responses received that have a similar theme.

In this particular example, we can see that the Return Rate: 9/106 of the respondents provided a comment or written response. In the far left hand column are the common themes that were extracted from the various comments. Some comments may have multiple themes that may be extracted and added to the tallied results. The following columns describe the number of times that particular theme occurred among the various comments, and of what percentage of this theme occurred among the comments. In the above example, the tool identified the theme “ Educator” nine times among the comments submitted. The following columns describe the individual sentiment of the comments for that specific theme. In this example, we can see that the theme “Educator” had an overall Positive sentiment. The sentiment score is a summary score that describes the overall positiveness or negativeness of a particular theme. The sentiment scores range from -1 to +1; positive sentiment scores are highlighted in green, no colour for neutral scores and negative sentiment scores are highlighted in red. In the example above, “Educator” had a neutral sentiment score of 0.09. The written responses to the open-ended questions can provide extremely helpful elaboration or explanation of the quantitative data. However, at times some comments can seem contradictory, with one student’s comment appearing to cancel out another’s. Therefore it is suggested that the quantitative and qualitative data be read in conjunction with each other.