Prototyping an Alternate Online Approach to Faculty Development

Saul CarlinerMonica Lopez

Yuan ChenYang GaoAudrey MariamoEzgi OzyonumDavid Price

Marianopolis College

3

Concordia – Marianopolis Collaboration… a bit of history

4

Concordia – Marianopolis Collaboration… a bit of history

§ Marianopolis Excellence in Teaching Profile (2013)§ Teacher with Students § Teacher with Subject Matter§ Teacher with Self§ Teacher with Peers, Institution and Community

5

Concordia – Marianopolis Collaboration… a bit of history

§ Common goals§ Provide PD resources online (evidence based)§ Promote faculty collaboration across institutions§ Showcase faculty pedagogical innovations: the Cracker

Barrel effect§ Interest in future research possibilities (PD in higher

education)

6

Concordia – Marianopolis Collaboration… a bit of history

§ Entente Canada-Quebec funding (2016-2017)§ Complete needs assessment (both institutions)§ Develop roadmap for the system§ Develop prototype resources

7

Design Framework: An Electronic Performance Support System (EPSS)

EPSSs provide workers with the resources they need to perform their job in the context of the job and at the moment of need

8

Design Framework: An Electronic Performance Support System (EPSS)

§ Manifestation of the concept of Human Performance Technology (Improvement) (Gery, 1995; Carliner, 2002; Gery, 2002; Mackenzie, 2002; Marion 2002).

§ Used in a variety of contexts (Gery, 1991):o Finance (Quick Books – accounting software)o Governmento Healthcare o Education (Early Literacy Tutoring)

9

Components of the proposed support systemInformative modules§ Each addressing a

particular teaching challenge identified by the faculty

§ Each module: o Synthesizes the

research on the topic—generally and in six broad disciplinary areas

o Summarizes individual studies

o Provides related case studies

Virtual meet-and-greetsTo further discuss the topics identified by the faculty

Regular e-mail messages and pollsAbout teaching-related topics to promote ongoing engagement with the system

The system

10

Needs Assessment - Methodology

§ Faculty Survey (both institutions)§ Online – 30 minutes§ Quantitative and qualitative analysis§ Comparisons across institution type, years of teaching,

class size§ Focus Groups

§ Random invitation§ 2 h sessions§ Qualitative analysis – Axial coding

11

Faculty Survey - Results

§ Faculty Survey - Demographics§ 112 participants (89 Concordia; 23 Marianopolis)§ Most respondents with 7+ years of experience§ Expected difference in maximum class size (41+ for

76% of Concordia faculty)§ Self reported teaching effectiveness (1 – 5)

§ 5 = My teaching is among the most effective in the institution

§ 1(3%) – 2(11%) – 3(24%) – 4(51%) – 5(11%)

12

Faculty Survey - ResultsPreparingacourseoutline(syllabus)Ensuringalignmentamongobjectives,activities,andassessmentDetermininghowmuchdetailtoincludeinacourseoutlineHomeworkDeterminingtheappropriateamountofreadingandhomeworkEnsuringthatstudentscompletereadingsforclassEnsuringthatstudentscompleteexercisesandassignments(otherthanreading)forclassClassdiscussionsGettingstudentstoparticipateinlargeclasses(60ormorestudents)Gettingstudentstoparticipateinmediumandsmallclasses(59orfewerstudents)Managingdiscussionsinlargeclasses(60ormorestudents)Managingdiscussioninmediumandsmallclasses(59orfewerstudents)GroupworkGettingteamstoproduceeffectiveworkAssessinggroupworkfairlyHelpingstudentsestablishpositivegroupdynamicsStudentnotetakingHelpingstudentstakeeffectivenotesAllowingdevicesintheclassroomsuchaslaptops,tablets,andsmartphones.TechnologyintheclassroomUsingPowerPointeffectivelyUsingstudentresponsesystems(suchas‘clickers’)effectivelyUsingMoodleeffectivelyAssessmentDeterminingtheappropriatenumberofassessmentsDeterminingtheappropriatetypeofassessmentsConstructingeffectiveandfairmultiplechoicetestsandquizzesConstructingeffectiveandfairessaytestsandquizzesIdentifyingalternativestoexams,essays,andquizzesCheating

13

Faculty Survey - ResultsMajor Challenges (from list)• Ensuring students complete readings before

class• Getting students to participate in class

discussions (classes of 60+ students)

Minor Challenges (from list)• Managing class discussions (all class sizes)• Determining appropriate amount of

homework• Ensuring students do their homework• Group work (group dynamics, effective

work, fair assessment)• Helping students with note-taking (major

challenge for college faculty)

Greatest Challenge (free text)• Overwhelming assessment• Unprepared students• Grade-focused students• Distracted and disengaged students (phone

and laptops)

Not a Challenge!• Power Point• Course Management Systems• Amount of detail in syllabus• Student Response Systems (clickers)

Faculty’s top source for teaching advice: Other faculty (ideally from their discipline)

14

Focus Groups- ResultsWhat faculty like about teaching• Constructive interactions with students• Influencing student learning (Aha!

moments)• Constant renewal, learning and updating • Justifying the ‘why’ of their courses

(usefulness)

No need, desire for extrinsic recognition (badges, credentials)

Teaching Challenges • Grade-focused students (commercialization

of education)• Overwhelming assessment (growing class

sizes)• Unprepared students (heterogeneous

ability)• Lack of coordination with colleagues

(departments/ programs)• Drag factors on innovation (bureaucracy,

budget cuts)• Research vs. teaching demands (lack of

time)

Main source of help and advice: Other faculty, locally and online; similar experiences and context; trusted, relevant and timely input.

15

Needs Assessment - Conclusion§ What we confirmed

§ Sense of agency and renewal fuels teaching§ Colleagues as main source of advice (trusted, relevant,

timely input)§ No need for extrinsic recognition§ Main challenges (from list): perennial teaching

problems§ What surprised us

§ Technology in the classroom, not a challenge!§ External factors vs. teachings tasks/behaviors as self

identified challenges

16

Development: Informative Modules§ Topics

§ Class Discussions§ Group Work§ Assessment - Multiple choice exams / Alternative

assessment§ ‘International students’

§ Literature Review§ 30-40 Peer-reviewed Articles (team ranking)§ 20 Structured Abstracts§ Quick Overview: Overall and in the disciplines

17

Development: Structured AbstractsResearch problem Summarize your purpose and rationale (1 to 2 sentences) Research questions:

Explicitly state the research questions

Literature review • Identify the bodies of literature you consulted • Summarize the key points of the review

Methodology • Identify your study as qualitative, quantitative, critical, or mixed

• Identify your study as case study, experiment, survey or other

• Describe how you chose participants and how many you used

• Describe how you chose your location and its type • Identify your method of data collection

• Name your analysis techniques Results and Conclusions

• Summarize your answers to the research questions • Summarize the implications of your results (1

sentence) • Summarize the limitations of your study (1 sentence) • Summarize your suggested future research (1

sentence)

18

Development: Quick Overview

19

Development: Teaching Cases§ Topics

§ Class discussions (English – Concordia)§ Use of graphics in concept attainment (Physics -

Marianopolis§ Applying essay feedback (English – Marianopolis)§ Ethics and social responsibility in engineering projects

(Engineering – Concordia)§ Process

§ Interview, transcription / template, validation (cycle)

20

Development: Teaching Cases

Teaching Case Template§ Background (challenge / goal / idea)§ The concept (instructional concept)§ The application (step by step implementation)§ About the instructor (short bio)§ Learn more about it (resources and related literature)§ Talk about it! (moderated comments)

21

Development: The prototype

22

Development: Integration with SALTISE / S4

The SALTISE / S4 project (Digital Professional Learning Community):

§ Supports faculty adoption of active learning pedagogies

§ Supports collaborations, co-design of materials and synergies between educational researchers, faculty, faculty developers and student support professionals

§ Connects members to other Quebec networks and institutional services (e.g. CTL, CSS, TLS)

23

Development: Integration with SALTISE / S4

Our contribution to SALTISE:§ Resources developed from faculty needs (perennial

teaching challenges) – pre-requisites for innovation§ Accessibility to research-based advise and literature

(structured abstracts); discipline specific§ Non-science and university level tools and examples

(Teaching Cases)

24

Development: Integration with SALTISE / S4

Integration Challenges§ Development a shared language (strategies, activities,

approaches)§ Smooth integration vs. project integrity§ Digital Professional Learning Community: can the

EPSS approach help?§ Researcher needs vs. practitioner needs (institutional

teaching and learning services)

25

What is Next?

§ Year 2 ECQ grant § Development of new Topics:

§ Possible focus on Assessment (Effective Essay Exams, Alternative Assessment)

§ Related Teaching Cases / Activities in the disciplines§ Pilot the interactive features of the system (increase

faculty engagement with the system)§ Test usability

26

What is Next? – Possible Research

§ What would be the faculty response to an EPSS regarding teaching issues?

§ How do responses differ among college and university instructors?

Thank you!

Monica Lopez m.lopez@marianopolis.eduSaul Carliner saul.carliner@concordia.caDavid PriceYuan ChenYang GaoEzgi OzyonumAudrey Mariamo

28

Consider this scenario.

A novice instructor who will be teaching his first undergraduate large-scale course at Bayshore University. During the new faculty orientation, he heard a lot of great stories about using group assignments and activities from his colleagues, and he learned that group activities could help students to learn effectively. He decided to add some group activities for his new course. While planning the activities, he realized that there are so many different types of group activities, and he was not sure which one to pick. He was confused, not sure where to start.According to him: “I want to use the group activities because they are fancy, and most of my colleagues are using them. But I don’t know where to start and what I can do with the activities.”

29

Also consider this scenario.You are a third-year STEM instructor who recently was called into the Ombudsperson’s office after a student filed a complaint about the “fairness” of the final exam. You have always prepared test questions based on what you expect students to know in the course, just as your professors have done in the past. The ombudsperson asked you to link each of your test questions to the objectives of your course. You could not do that. The ombudsperson asked you to explain how the incorrect responses (which he called distractors) were generated. You could not explain that either.The ombudsperson determined that you need to rewrite your final exam. You do not want anyone to know that this has happened but you realize you have no idea of how to write a multiple choice exam that would withstand this level of evaluation.

30

Consider this scenario, too.You are a history instructor. This term, you are teaching a second-year course on Canadian history. Feeling exacerbated after the second class, you complain to a colleague about your students having no awareness of John A. Macdonald and Louis Riel. As you mentioned to your colleague, “After mentioning Wayne Gretzky in class, one of my students even shamelessly asked in class, “Who is she?”“SHE?” you said to your friend.“Well, where are they from?” your colleague asked. After she corrected you when you said Greece (no, she didn’t mean Plato and Aristotle), you mentioned that of your 50 students, only 25 were originally from Canada. Ten are international students (primarily from China), another 8 were recent immigrants from North Africa, and the rest are from Iran. “That’s the issue,” your colleague said. “Your students aren’t stupid. You’re just making some false assumptions about their previous education.”

31

EXTRA SLIDE Journey the instructor takes to seeking assistance§ Wow! I wanna do a group assignment§ Seek help—know where to start. Google that unless I know of

something already. § Which learning goals I want to use groupwork for. § What type of groupwork do I wanna do. Types—ice breaker, § Design the group work§ How to run the group work if I’m new. How to put students into groups. § How to make them engage in the group work or start groupwork or make it

interesting § Class time and try it. Observe. § How did it go? Not great. Students won’t discuss. Not prepared. § Ask for student feedback? Maybe receive complaints from students that the

activity is stupid or dumb. § Ask others for suggestions. § Change my plan afterwards.