AN ANALYSIS OF FIFTEEN YEARS OF THE NATIONALEFFECTIVE TEACHING INSTITUTE*

Richard Felder, North Carolina State University, rmfelder@mindspring.comRichard Felder is Hoechst Celanese Professor Emeritus of Chemical Engineering at NorthCarolina State University. He is a Fellow Member of ASEE, a founding director of the ASEENational Effective Teaching Institute, and the recipient of the ASEE Chester F. Carlson Awardfor Innovation in Engineering Education and the ASEE Chemical Engineering Division LifetimeAchievement Award for Pedagogical Scholarship.

Rebecca Brent, President, Education Designs Inc., Cary, NC, rbrent@mindspring.comRebecca Brent is an education consultant and licensed program evaluator specializing in facultydevelopment for effective university teaching, K-12 staff development in language arts andclassroom management, and effective uses of technology in K-12 and university classrooms. Shecoordinates faculty development activities for the College of Engineering at North Carolina StateUniversity, and since 1996 she has been a codirector of the ASEE National Effective TeachingInstitute. She is a former Associate Professor of Education at East Carolina University.

*Proceedings of the 2009 ASEE Annual Conference. Washington, DC: ASEE (June 2009).

ANALYSIS OF FIFTEEN YEARS OF THE

NATIONAL EFFECTIVE TEACHING INSTITUTE

Key Words: National Effective Teaching Institute, NETI, Faculty Development

ABSTRACT

The National Effective Teaching Institute (NETI) is a three-day teaching workshop that

has been given annually since 1991 in conjunction with the Annual ASEE Conference. In the

early spring of 2008, a web-based survey sent to 607 workshop alumni asked about the effects of

the NETI on their teaching practices, their students’ and their own ratings of their teaching, their

involvement in educational research and instructional development, and their attitudes regarding

various aspects of teaching and learning. Valid responses were received from 319 of the survey

recipients. This paper briefly reviews the history of the NETI, summarizes and analyzes the

survey responses, and discusses their implications for engineering faculty development.

INTRODUCTION

The National Effective Teaching Institute is given annually on the Thursday, Friday, and

Saturday preceding the Annual Conference of the American Society for Engineering Education.

It is sponsored by the Engineering Research and Methods and Chemical Engineering Divisions

of the ASEE, and the ASEE program staff manages the finances, registration, and logistical

arrangements. Every January, all deans of engineering and engineering technology in the U.S.

are invited to nominate up to two of their faculty members for the NETI, and applications are

accepted on a first-come-first-served basis up to a maximum of 55. In the years 1991–2008, the

workshop has been attended by 935 professors from 209 different schools (Appendix A).

Information about the NETI can be found at <http://www.ncsu.edu/felder-public/NETI.html>.

Topics covered in the NETI include designing instruction to address the full spectrum of

student learning styles; planning courses (including writing learning objectives covering all

cognitive levels of Bloom’s Taxonomy); assessing and evaluating learning; effective lecturing;

active learning; teaching problem-solving skills; time management; and dealing with a variety of

problems that commonly arise in the careers of engineering educators. Cooperative learning and

inductive teaching methods such as inquiry-based learning and problem-based learning are

introduced but minimal instruction in them is given. During the afternoon of the second day, two

parallel 90-minute sessions are held: one for relatively new faculty members on getting academic

careers off to a good start, and one for more experienced faculty members on techniques for

promoting effective teaching on individual campuses. For each topic addressed in the workshop,

practical suggestions are offered and the research attesting to their effectiveness is cited and

discussed.

Participant evaluations collected at the conclusion of each workshop offering have been

consistently positive. In the eighteen years that the workshop has been given, 820 overall ratings

have been submitted of which 84% were “Excellent,” 16% were “Good,” <1% were “Average,”

and none were “Fair” or “Poor.” As gratifying as this level of participant satisfaction is, however,

it provides no real indication of what the workshop has actually accomplished. In the spring of

2008, we designed and administered a survey to all of the NETI participants in the 1993–2006

offerings whose contact information we could find. The survey—hereafter referred to as the

NETI Alumni Survey—is shown in Appendix B. It asks the participants about their teaching

practices, their students’ and their own ratings of their teaching, the effects of the NETI on both

their practices and their ratings, their involvement in educational research and instructional

development, and several demographic questions. The survey was administered to 607 NETI

alumni, of whom 319 submitted usable responses for a 53% rate of return. Three of the questions

on the Alumni Survey were duplicates of questions in pre-workshop surveys administered to the

2005 and 2006 NETI participants, so a direct assessment could be made of changes in those

participants’ teaching practices and attitudes from just before they participated to two or three

years afterwards.

This paper briefly reviews the history of the NETI, offers some general considerations

regarding evaluation of faculty development programs, describes the design and administration

of the Alumni Survey and the pre-workshop surveys, summarizes and analyzes the responses,

and discusses their implications for engineering faculty development.

A BRIEF HISTORY OF THE NETI

For several years prior to 1988, the term “National Effective Teaching Institutes” had

been applied to half-day and one-day workshops on various topics sponsored by the Educational

Research and Methods Division of the ASEE and given at ASEE Conferences. At the 1988

ASEE Conference in Portland, Oregon, James Stice of the University of Texas and Richard

Felder of North Carolina State University had the idea that a more extended program offered

every year to engineering faculty at large—not just conference attendees—might have a greater

impact on American engineering education and might also induce more engineering educators to

join ASEE. Felder had been giving three-day workshops at N.C. State University since 1986, and

the two chemical engineering professors decided to use a 3-day format similar to the one used at

N.C. State. They sought and received agreement from the ERM and CHE Divisions of ASEE to

sponsor the NETI, and Stice solicited and received financial support from several companies

(Dow Chemical, Dupont, Exxon-Mobil, and Union Carbide).

Early in 1991, all engineering deans on the ASEE membership list were invited to

nominate up to two of their faculty members to attend the NETI, and the first offering was given

to about 50 attendees at the 1991 ASEE Annual Meeting in New Orleans. The first presenters

were Stice, Felder, and Rebecca Leonard, a professor of communication at N.C. State. That

offering took place immediately following the conference, and all subsequent offerings have

been on the Thursday, Friday, and Saturday preceding it. The sponsoring companies maintained

their support for varying lengths of time, and starting in 2008 the NETI has supported itself

entirely from participant registration fees.

Stice and Felder have co-facilitated every NETI since the inception of the program.

Becky Leonard presented from 1991 through 1994, and beginning in 1996, Stice and Felder were

joined by Rebecca Brent, a former professor of education at East Carolina University and now

president of Education Designs, Inc., an educational consulting firm in Cary, North Carolina. In

2001, with a view toward creating a cadre of teaching leaders who could disseminate the

methods being taught in the NETI throughout the engineering education community, Stice,

Felder, and Brent created the “NETI Fellow” program, inviting rising stars of the field to co-

present with them each year. In 2002, P.K. Imbrie of Purdue and Mike Prince of Bucknell served

as the inaugural NETI Fellows. Subsequent fellows were Lisa Bullard of N.C. State, Kay C. Dee

of Rose-Hulman, Gary Huvard of Virginia Commonwealth, Glen Livesay of Rose-Hulman, and

Matt Ohland of Purdue.

Jim Stice has announced his intention of stepping down as NETI co-director after the

2009 offering in Austin, but the other members of the team have persuaded him to accept the

post of Director Emeritus, in which capacity he will continue on occasion to share his wisdom

with the NETI participants and presenters. In anticipation of Jim’s retirement, Mike Prince was

invited to become a permanent co-director of the NETI, a post he assumed at the 2008 NETI in

Pittsburgh. The NETI Fellow program will resume in 2010.

SURVEY DESIGN AND ADMINISTRATION

Evaluating Teaching Workshops

Van Note Chism and Szabó1 observe that evaluation of a teaching workshop (or any other

faculty development service) can be performed at three levels defined by certain focus questions:

• Level 1: How satisfied were the participants with the workshop?

• Level 2: What was the impact of the workshop on the participants’ teaching?

• Level 3: What was the impact of the workshop on the participants’ students’ learning?

The ultimate goal of teaching is learning, and so the ultimate measure of the effectiveness of a

teaching workshop is the improvement in the participants’ students’ learning that can be

attributed to the participants’ having attended the workshop (Level 3). Improvements in students’

learning cannot be assumed to follow from their teachers’ satisfaction with a workshop (Level

1), and may only be inferred indirectly from changes in the teachers instructional practices

following workshop attendance (Level 2). The Level 3 question is therefore the one that really

matters in our evaluation of the NETI, and if we could get an unequivocal answer to it there

would be no need to ask the other two questions.

Unfortunately, we cannot get that answer. There is no way to retrospectively assess the

learning of students taught at 100–200 universities during a 15-year period by several hundred

engineering professors before and after they attended the NETI—and even if it could be done,

there would be no way to determine how much of any observed learning outcome improvement

could be attributed to the workshop. We must therefore resort to indirect evaluation, asking

Question 2 and using the survey respondents’ appraisals of how the NETI affected their teaching

and their students’ learning to speculate on the answer to Question 3.

In adopting this approach, we are following standard practice in faculty development

program evaluation. When teaching workshops are evaluated at all, the evaluation generally

consists of surveying the participants immediately afterwards, asking them to rate the workshop

and the presenters on some numerical or verbal scale and perhaps to comment on things they

liked and disliked (Level 1 assessment). Van Note Chism and Szabó1 found that 85% of the

faculty development programs on the 200 campuses they surveyed assessed at this level. It is of

course important to assess participant satisfaction to identify problems and obtain guidance on

how to improve subsequent offerings, but satisfaction surveys provide no indication of the

subsequent impact of the workshops on either teaching or learning. Fewer than 20% of Van Note

Chism and Szabó’s respondents indicated that they always or usually evaluate the impact of their

services on teaching (Level 2), and virtually none have attempted to evaluate impact on students’

learning.

The construct validity of using participants’ self-assessments of their teaching for

workshop evaluation has been examined by D’Eon et al.,2 who cite a number of studies that

compared self-assessments of teaching with external evaluations by trained observers. Those

studies support two conclusions:

• An individual’s assessment of his or her teaching skill before or after a workshop cannot

be taken at face value, but aggregated self-assessments from workshop participants

generally match closely with external assessments and can provide the basis for a valid

and reliable evaluation of workshop effectiveness.

• Individual gains in skill calculated from separate pre-workshop and post-workshop

assessments are also suspect, since before the workshop individuals often lack a

legitimate basis for judging their skill levels. On the other hand, individuals’ retrospective

(post-workshop) self-assessments of pre-post workshop gains in skill levels correlate

reasonably well with more objective external ratings.

The Alumni Survey called on past NETI participants to retrospectively assess the impact of the

workshop on their teaching practices and quality, and their responses along with their statements

about the impact of the workshop on their student ratings were aggregated and used to evaluate

the effectiveness of the NETI. The validity of this approach is supported by the conclusions of

D’Eon et al.2

Structure and Administration of the NETI Alumni Survey

The original goals of the NETI were to improve the participants’ teaching, to persuade

some of them to pass on what they learned in the NETI to colleagues on their home campuses

(i.e., to engage in instructional development), and to encourage them to join ASEE. Inspired by

the publication of Boyer’s Scholarship Reconsidered3 and subsequent work at the Carnegie

Foundation for the Advancement of Teaching by Hutchings and Shulman,4 we expanded our

goals to include promoting scholarly teaching and the scholarship of teaching and learning. The

Alumni Survey was designed to assess how well each of these goals was met in the 1993–2006

offerings of the NETI.

The survey, which is shown in its entirety in Appendix B, addresses the following

questions:

1. In what ways and to what extent has the NETI improved its participants’ teaching?

Questions relate to students’ ratings of the participants’ teaching in undergraduate and

graduate courses, how the ratings changed in the years following NETI attendance, and

the participants’ opinions about the effects of the NETI on their ratings and their teaching

irrespective of ratings (Questions 7–11 and 20); the participants’ awareness and use of

selected teaching concepts and strategies and the effect of the NETI on their

incorporation of those concepts and strategies into their teaching (Questions 14 and 15);

and their agreement or disagreement with several teaching practices advocated in the

NETI and several common misconceptions that the NETI challenges (Question 19).

2. To what extent has the NETI motivated participants to engage in instructional

development? Question 12 asked the participants to state whether they had ever provided

instructional development and asked those who had not done so whether they intended to

provide it in the future, and Question 13 asked them whether the NETI motivated them to

become involved in instructional development.

3. Did the NETI motivate participants to join the ASEE? Question 18 asked whether the

participants were members of the ASEE and whether the NETI motivated them to join.

4. Has the NETI promoted scholarly teaching and the scholarship of teaching and learning?

Questions 5, 6, 16, and 18 asked (a) whether participants had engaged in practices that

characterize scholarly teaching (reading education-related papers, attending education-

related seminars, workshops, and conferences, using classroom research to assess the

effectiveness of their teaching, and reflecting on and attempting to understand the

processes of teaching and learning in general and their students’ learning strengths,

weaknesses, and preferences), and (b) whether they had engaged in the scholarship of

teaching and learning (publicly presenting and/or publishing results of educational

research studies). Question 17 asked whether the NETI motivated the participants to do

research on teaching and learning.

The remaining survey items provided demographic information about the respondents.

We designed the Alumni Survey early in the spring of 2008 and concurrently made an

effort to get as complete a list as possible of NETI alumni names and current email addresses. No

participant lists could be found from the 1991 and 1992 offerings of the workshop, and we chose

not to ask the 2007 participants to evaluate the impact of a workshop they had taken only six

months earlier. We initially located 692 email addresses from the lists for the 1993–2006

offerings, sent announcements of a forthcoming NETI impact survey, and gave the recipients the

option to not participate. (Five of them exercised that option.) Many of those messages were

returned as undeliverable. We then searched the Web to try and find current email addresses for

the recipients of the returned messages, and in many cases we were successful and re-sent the

survey notification to verify that new addresses were correct.

Once we had a usable list of emails, we administered the survey using Survey Monkey®

(<www.surveymonkey.com>). We sent reminder emails to alumni who had not responded within

two weeks of receiving the survey, and a second set of reminders to non-respondents after

another three weeks. Tables 1 and 2 show the final response statistics. Considering the fact that

we were surveying individuals with whom we last had contact as much as 15 years earlier, we

were pleased that we were able to obtain usable e-mail addresses for 88% of the NETI

participants and that 53% of those receiving surveys completed and returned them. Sue and

Ritter5 report an analysis of e-mail surveys showing response rates that varied from 24% to 76%,

and they and Babbie6 suggest that above 50% is acceptable for analysis and publication and 60%

is good.

Table 1. Overall survey response statistics

(a) Number of NETI participants in 1993–2006 731

(b) Number of participants with usable email addresses 607 [83% of (a)]

(c) Number of valid responses 319 [53% of (b)]

Table 2. Responses by year of NETI attendance

Year (N)attending (N)surveyed (N)responding

1993 51 39 17 (44%) 1994 48 37 20 (54%) 1995 49 38 14 (37%) 1996 57 44 22 (50%) 1997 52 44 22 (50%) 1998 54 43 27 (63%) 1999 56 38 19 (50%) 2000 56 42 20 (48%) 2001 52 48 31 (65%) 2002 47 41 23 (56%0 2003 46 41 16 (39%) 2004 56 51 25 (49%) 2005 53 49 31 (63%) 2006 54 52 32 (62%)

Total 731 607 319 (53%)

Administration of the Pre-Workshop Surveys

Beginning in 2005, all NETI participants have been asked to complete an online survey

before coming to the workshop, both to provide demographic information and to furnish a basis

for eventual comparisons of pre-workshop and post-workshop teaching practices, attitudes, and

student ratings. Questions 7, 14, and 19 of the Alumni Survey (Appendix B) are replicates of

questions in the pre-survey.

Fifty-one workshop participants completed both surveys—23 in 2005 and 28 in 2006. A

paired sample t-test was used to test for significant differences between pre-workshop and post-

workshop student ratings (Question 7 of the Alumni Survey). The letter responses (a–e) to

Questions 14 and 16 were converted to numbers (1–5), and Wilcoxon Matched Pairs Tests were

used to analyze pre-post response differences in awareness and use of specified concepts and

teaching strategies (Question 14) and differences in agreement or disagreement with certain

beliefs about teaching (Question 19). The statistical tests were carried out using StatPlus:mac®

and Excel®

.

The pre-workshop surveys were administered to all registered participants using Survey

Monkey about a month before the workshops were given, and one reminder was sent several

weeks later to non-respondents. A paper version of the survey was given at the beginning of each

workshop to everyone who had not responded electronically. Most paper surveys were

completed and turned in during the preliminary registration period, and the remaining ones were

collected at the first coffee break a little over an hour after the workshop began.

SURVEY RESPONSES

In the remainder of this paper, references to “the survey” and to survey responses should

be understood to refer to the NETI Alumni Survey shown in Appendix B, unless specific

mention is made of the 2005 and 2006 pre-workshop surveys.

Demographics

Table 3 summarizes responses to the survey items related to the demographics of the

respondents. Since most of the alumni surveyed attended the NETI many years previously, it is

not surprising that 68% of the respondents were full or associate professors. Eighty-one percent

were at Ph.D.-granting institutions and 69% were currently involved in research. The percentage

who had been involved with research at any time in their careers would be greater, since it would

also include faculty who had moved into administration or ended their research programs for

some other reason.

Table 3. Demographics and experience of respondents

N Responses Current position 302 full prof.–26%, assoc.–42%, asst.–21%,

lecturer/instr.–2%, grad. stud.–0%, other–9% Gender 302 female–19%, male–81% Current inst. classification 319 RU/VH (Res. I)–47%, 4 yr. univ. with Ph.D.–34%,

4-yr. univ. w/o Ph.D.–17%, tech./community coll.–0.3%, other–2%Principal responsibilities 302 research–69%, undergrad. teaching–85%, grad. teaching–58%,

administration–32%, advising/counseling–42%, instructional development–18%, other–7%

Years taught 319 (<1)–0%, (1-2)–0.3%, (>2-5)–13%, (>5-10)–26%, (>10)–61% Years taught before NETI 302 (<1)–17%, (1-2)–33%, (>2-5)–22%, (>5-10)–13%, (>10)–15% Av. annual teaching load in past two years (courses/yr)

319 (0)–4%, (1–2)–18%, (>2–4)–48%, (>4–6)–21%, (>6)–9%

Years worked in industry not counting internships

319 (0)–34%, (<1)–8%, (1-2)–12%, (>2-5)–18%, (>5-10)–14%, (>10)–13%

We are under no illusion that the survey respondents are representative of American

engineering educators as a class: the respondents are, after all, people who volunteered to attend

a 3-day workshop on effective teaching. We also have no way to determine the extent to which

the 319 individuals who completed and returned surveys are representative of the 731 who

attended the NETI or the 607 who were sent surveys. Our purpose, however, is not to draw

general inferences about engineering faculty members from the survey results; it is simply to

investigate the impact of a long-standing engineering faculty development program on its

participants.

Participants’ Open-Ended Statements about Impacts of the NETI on their Teaching

Question 11 of the survey asked the respondents to comment on the effect of the NETI on

the quality of their teaching, irrespective of student ratings, and 295 of them responded. The

responses were tabulated and sorted into categories that emerged in the course of the tabulation,

with some responses falling into two or more categories. A summary is given in Table 4.

Of the 412 entries in Table 4, 131 reflected perceptions that the NETI had a positive

effect on the quality of the respondents’ teaching and/or of their students’ learning; 12 noted

feelings of greater confidence (or comfort or control) in teaching; and 22 indicated that the NETI

either had a slight or negligible effect on teaching quality or that the respondents did not know or

could not remember its effect. In short, 87% of the 165 direct assessments of the effect of the

workshop on teaching quality were clearly positive, 13% were not positive, and none were

clearly negative. Several respondents noted post-NETI drops in their teaching ratings but

expressed beliefs that their teaching had nonetheless improved. In their opinion, the rating

declines occurred because their teaching had become more challenging and learner-centered after

the NETI and some of their students did not welcome the change. This high overall level of

satisfaction with the NETI is consistent with the previously cited post-workshop ratings (84%

excellent, 16% good, <1% average, and none fair or poor).

The remaining 281 responses to Question 11 noted specific benefits that resulted from

NETI participation. The most frequently named benefit (131 entries) was learning new teaching

methods or getting better at applying previously used methods. Of the methods specified, active

learning (active engagement of students in class) was cited most often (41), followed by the

systematic use of learning objectives and/or Bloom’s Taxonomy (21). Responses to closed-ended

questions to be summarized shortly suggest that the number of participants who actually learned

about and adopted new strategies was much greater than the number who thought of mentioning

them in response to Question 11.

The next most common category of responses to Question 11 had to do with the

respondents’ improved understanding of students (62 responses)—their greater awareness of

students’ learning needs and differences in their learning styles (a major topic in the NETI) and

greater incorporation of those needs and differences into the respondents’ teaching (30), greater

learner-centeredness in their teaching (21), and a generally better understanding of and rapport

with students (11).

Most college instructors are never taught about teaching and so simply repeat what they

saw their own college instructors do, without reflecting on the effectiveness of that instructional

approach in promoting learning.7 Twenty-five responses to Question 11 indicated that the NETI

made the respondents more reflective in their teaching practice, with 14 indicating that they had

become more aware of teaching and learning processes in general and 11 stating that they had

begun to think more consciously about their own teaching. As was the case with teaching

strategies, the number of participants who actually became more reflective practitioners was

undoubtedly much greater than the number who thought to mention it explicitly in their open-

ended responses.

Table 4. Effect of NETI on Teaching (Open Responses)

Sample Comments

Affected quality positively (131)

Helped teaching, improved teaching a

lot/moderately, improved student learning

- The pedagogy was invaluable as were the assessment techniques that I still use.

- One year after taking NETI, I received our College Outstanding Teacher Award.

- Most significant factor in my success.

- This was the best thing I could have done at the start of my tenure track. Thanks.

- NETI changed my life and probably saved my career.

More confident, comfortable, in control (12) [NETI] gave me more confidence to experiment with new or different teaching techniques.

Better organized/more efficient (6) Helped me to organize lectures better, therefore use time much more efficiently.

More creative (2) I was encouraged to use my creativity to approach teaching with innovation [as I approach] research.

Affected quality slightly or not at all, or effect

unknown (22)

- I had seen (and applied and taught) most of the material covered in NETI already.

- It has been so long, it is hard to say.

Learned/improved teaching methods (131)

Unspecified (46)

Active learning/engagement of students (41)

Learning objectives/Bloom’s Taxonomy (21)

Collaborative/cooperative learning (9)

Assessment, testing & grading (8)

Course preparation (4)

Research on teaching (2)

- Helped me identify strategies for increasing student mastery of concepts.

- My motto changed to “No more than 15 minutes of lecture without some active learning exercise.”

- Was very helpful in matching course objectives with student deliverables.

- It opened my eyes to cooperative learning.

- Improved the way I write homeworks and exams.

- It improved the way I plan a course, taking into account students and the program.

- It exposed me to the opportunities to practice the scholarship of teaching.

Improved understanding of students (62)

Aware of/use learning/teaching styles (30)

More learner-centered (21)

Better understanding of /rapport with students (11)

- I became aware of different learning styles and how to effectively address each of them.

- Classes are much more student learning focused than they would otherwise be.

- I do think it helped me develop a better appreciation for the students’ perspective.

More reflective about teaching (25)

More aware of teaching & learning processes (14)

More reflective about own teaching (11)

NETI essentially made me really think about teaching styles and approaches.

It began to get me thinking seriously about what I do in the classroom and why.

Improved mentoring/faculty development (6) I attended NETI to enhance my skills as a workshop planner and faculty mentor. IT WAS GREAT—

and gave me the courage of my convictions.

Reinforced what I was already doing (6) I am certain that quality went up [but] for me the workshop was more a matter of reinforcing and

organizing things in my mind.

Helped network with colleagues (5) Confirmation that there was a community of engineering professors interested in teaching.

Gave research/theory backup for recommended

teaching strategies (4)

Confirmed that my teaching style had a scientific background to prove that what I was doing was the

right thing to do.

Pedagogical Knowledge and Practices

An obvious goal of a teaching workshop is that the participants learn some things—new

planning or teaching or assessment techniques, and perhaps mistakes they have been making—

that lead them to make changes in how they teach. As long as those changes have been shown by

research to correlate with positive learning outcomes, making the changes can reasonably be

presumed to lead to greater student learning.

Question 14 of the survey asked the alumni to rate their awareness and use of certain

concepts and teaching strategies, and Question 15 asked them to judge the impact of the NETI on

their incorporation of those concepts and strategies into their teaching. Not all of the concepts

and strategies listed were focal points of the NETI. Some, such as learning styles and learning

objectives, were major workshop topics; others, such as inquiry-based instruction, were

addressed peripherally; and still others, such as distance education and Web-based tutorials, were

barely mentioned in the NETI. The frequency distributions of the responses are summarized in

Table 5. Pre-workshop survey responses to Question 14 were submitted by 51 participants in the

2005–2006 NETI offerings, and Table 5 also summarizes the average pre-post workshop

response differences.

The first topic discussed in the workshop is learning styles of engineering students,

teaching styles of engineering professors, and consequences of mismatches. Ninety-one percent

of the survey respondents indicated that they had made occasional or frequent use of the concept

in their teaching, and 68% reported that the NETI had a moderate or substantial effect on their

having done so. The difference between pre-workshop and post-workshop awareness and use of

learning styles in teaching was statistically highly significant. That only 30 of the 412 responses

summarized in Table 4 (7%) explicitly mentioned learning styles supports our argument that

open-ended response frequencies seriously underestimate the true effects of the NETI on

teaching practices.

The topic after learning styles is using learning objectives as the basis for designing

instruction and assessment. Learning objectives (also known as instructional objectives) are

explicit statements of what students should be able to do (define, explain, calculate, derive,

model, design, critique,...) if they have learned what the instructor intends them to learn. Bloom’s

Taxonomy is a hierarchical system used to classify learning objectives in increasing order of

complexity, from pure rote memorization at Level 1 through basic conceptual understanding

(Level 2), ability to apply concepts and procedures to solve new problems (Level 3), analytical

thinking (Level 4), critical thinking (Level 5), and creative thinking (Level 6). James Stice—who

introduced the concept of learning objectives to engineering education in a landmark paper8—

leads a NETI session on objectives and Bloom’s Taxonomy, and the participants are given

practice in writing objectives at different Bloom levels for a course they teach.

Because of their prominent role in the ABET Engineering Criteria, learning objectives

are likely to be familiar to engineering professors whether or not they attended the NETI, and

98% of the respondents indeed reported making occasional or frequent use of them, but 75%

stated that the NETI had a moderate or substantial impact on their doing so. Bloom’s Taxonomy

(which is not part of the ABET criteria) was made use of by 73% of the respondents, with 55%

reporting that the NETI had a moderate or substantial impact on their use.

Table 5. Pedagogical Knowledge and Practices

Concept or Strategy

Role in NETI

A Awareness & Use

B Impact of NETI on Use

C Pre-Test ResponseMean (SD)

D Post-Test ResponseMean (SD)

D p

E

Learning styles

(a) (1)–0%, (2)–0.6%, (3)–8%, (4)–48%, (5)–43%

(1)–9%, (2)–24%, (3)–33%, (4)–34%

3.59 (0.96) 4.30 (0.69) .0002***

Learning objectives

(a) (1)–0.3%, (2)–1%, (3)–1%, (4)–21%, (5)–77%

(1)–7%, (2)–18%, (3)–32%, (4)–43%

4.24 (0.86) 4.67 (0.62) .005**

Bloom’s Taxonomy

(a) (1)–2%, (2)–7%, (3)–17%, (4)–46%, (5)–27%

(1)–26%, (2)–19%, (3)–33%, (4)–22%

2.67 (1.47) 3.90 (1.00) <.0001***

PowerPoint (or equivalent)

(c) (1)–0%, (2)–0%, (3)–12%, (4)–47%, (5)–40%

(1)–56%, (2)–23%, (3)–16%, (4)–5%

4.39 (0.70) 4.37 (0.67) .85

Active learning

(a) (1)–0%, (2)–0%, (3)–2%, (4)–41%, (5)–57%

(1)–6%, (2)–20%, (3)–34%, (4)–40%

3.67 (1.16) 4.49 (0.58) <.0001***

Cooperative learning

(a) (1)–0%, (2)–2%, (3)–7%, (4)–51%, (5)–39%

(1)–13%, (2)–22%, (3)–37%, (4)–28%

3.43 (1.22) 4.06 (0.76) .0003***

Problem-based learning

(c) (1)–0.3%, (2)–2%, (3)–5%, (4)–44%, (5)–49%

(1)–20%, (2)–26%, (3)–37%, (4)–18%

3.76 (1.03) 4.18 (0.74) .014*

Inquiry-based learning

(b) (1)–2%, (2)–12%, (3)–16%, (4)–48%, (5)–22%

(1)–39%, (2)–26%, (3)–26%, (4)–8%

2.76 (1.11) 3.69 (0.95) <.0001***

Web-based tutorials

(c) (1)–0.3%, (2)–0.3%, (3)–55%, (4)–33%, (5)–11%

(1)–74%, (2)–16%, (3)–9%, (4)–1%

3.37 (0.60) 3.35 (0.74) .82

Distance education

(c) (1)–0%, (2)–0.6%, (3)–71%, (4)–18%, (5)–10%

(1)–88%, (2)–8%, (3)–3%, (4)–1%

3.14 (0.57) 3.27 (.63) .15

Boice’s new faculty strategies

(b) (1)–39%, (2)–27%, (3)–16%, (4)–14%, (5)–4%

(1)–83%, (2)–8%, (3)–7%, (4)–2%

1.75 (1.09) 2.75 (1.31) <.0001***

Legend

A. (a)–major topic, (b)–moderately addressed, (c)–slightly addressed

B. N=310. (1)–never heard of, (2)–not sure what it is, (3)–never use, (4)–use occasionally, (5)–use frequently

C. N=310. (1)–no effect, (2)–slight effect, (3)–moderate effect, (4)–substantial effect

D. N=51. (1)–never heard of, (2)–not sure what it is, (3)–never use, (4)–use occasionally, (5)–use frequently

E. Significance level from Wilcoxon’s Matched Pairs Test. *=significant at .05 level, **=significant at .01 level, ***=significant at .001 level

The third main focal point of the NETI is active learning (getting students involved in

class doing course-related activities other than watching and listening to a lecture). It is used

throughout the workshop and addressed formally in two separate sessions: one led by Felder and

Brent on different active learning structures and strategies for making implementation of active

learning as effective as possible, and one led by Stice on using the active learning strategy called

Thinking-Aloud-Pair-Problem-Solving (TAPPS)9 to help students develop problem-solving

skills. Our experience is that most faculty members who come to the NETI have a limited and

frequently incorrect idea of what active learning is, but 98% of the respondents reported making

occasional or frequent use of it and 74% credited the NETI with having played a moderate or

substantial role in their doing so. Further evidence of the influence of the NETI is provided by

the pre-post test results in Table 5, which show a highly significant increased awareness and use

of active learning following the NETI.

The survey also showed that the NETI motivated many participants to adopt or increase

their use of cooperative learning (team-based learning under conditions that include holding all

team members individually accountable for the complete content of the team assignments or

projects), with 90% of the respondents reporting occasional or frequent use of it and 65%

acknowledging the moderate or substantial impact of the NETI in their decision to do so. The

2005 and 2006 NETI participants who completed pre-workshop surveys also reported a dramatic

increase in their awareness and use of cooperative learning following their participation.

The results shown in Table 5 for problem-based learning and inquiry-based learning

should be taken with a grain of salt. While both of those approaches are mentioned favorably in

the NETI, we do not really “cover” them in the detailed way we do with active and cooperative

learning. Thus, while 93% and 70% of the respondents respectively said they use problem-based

and inquiry-based learning, and the pre-post workshop gains in awareness and use were

significant for both strategies, we strongly suspect that experts on those inductive methods would

not qualify what many of those respondents are doing as either PBL or inquiry. The low impacts

of the NETI reported in Table 5 on the use of Web-based instruction and distance education—

topics which the NETI does little more than touch on—are undoubtedly more realistic.

The unimpressive results in Table 5 for the use of Robert Boice’s10

success strategies for

new faculty (budget regular time for writing, avoid overpreparing for classes, network regularly

with colleagues, and do long-term strategic career planning) are understandable. We only started

presenting this material when we initiated a breakout session for new faculty members in 2002,

so respondents who attended the NETI before that year and respondents who attended the other

breakout session in the 2003–2006 offerings would not have seen it. Also, the strategies are not

nearly as concrete and easy to adopt as are such things as writing learning objectives and getting

students actively involved in class. Boice found that it took a 12-week program to induce and

prepare new faculty members to adopt his strategies. Nevertheless, roughly 10% of the

respondents found that the NETI had a moderate or substantial influence in motivating them to

make use of the strategies, and the pre-post survey results (which are only applicable to recent

participants who experienced more extensive coverage of the Boice material than their

predecessors did) show a clear impact of the NETI on the respondents’ awareness and use of the

strategies.

In short, the results summarized in Table 5 suggest that the NETI was successful in both

increasing the participants’ awareness of the main concepts and instructional methods discussed

in the workshop and persuading them to incorporate some of those concepts and methods into

their teaching. Since correlations with improved student learning have been found for teaching to

balance the needs of students with different learning styles,11

writing explicit learning objectives

and making them clear to students,12

getting students actively engaged in course-related activities

in class,13

and getting students to work in teams under conditions that assure individual

accountability and the other defining criteria of cooperative learning,14

we infer that the NETI-

motivated adoption or increased use of these teaching strategies by many participants led to

greater learning by their students.

Student Ratings of Teaching

Student ratings alone are indicators but not self-sufficient measures of teaching quality.

They have, however, been repeatedly shown to correlate well with peer ratings and other quality

metrics,15

and students are uniquely qualified to evaluate the extent to which instructors display

certain attributes of teaching effectiveness (e.g., motivating students, holding their interest, being

available and willing to help them outside class, etc.). The survey asked the respondents to

estimate average student ratings for undergraduate and graduate courses they taught in the

previous two years, with 5=excellent, 3=average, and 1=poor. There were 287 valid responses

for undergraduate ratings and 205 for graduate ratings. The average undergraduate rating was 4.1

(SD=0.6) and the average graduate rating was 4.4 (SD=0.4). Figure 1 shows histograms of the

results. Although we cannot say anything definitive about the respondents’ teaching based only

on those results, it appears that there were relatively few poor teachers among them, and that on

average most were good to excellent.

A common fear of instructors is that if they start using learner-centered teaching methods,

some of their students won’t like it (which is true enough) and their ratings will suffer. The

possibility of decreased ratings certainly exists, especially for outstanding instructors who are

already getting the highest ratings possible. To investigate the link between NETI attendance and

decreased ratings, we asked the survey respondents to indicate what happened to their ratings in

the years following their attendance. Six of the respondents skipped this question and 17 said that

it was inapplicable to them (perhaps because they did not do any teaching in the years after the

NETI). The distribution of the remaining 290 responses is shown in Figure 2.

Clearly, the feared drop in ratings was the exception and not the rule. Only one of 290

respondents reported a substantial drop following the NETI and fewer than 6% reported a drop

of any magnitude at all. For 29% the ratings neither decreased nor increased, and 67% reported

increases. We also asked the participants to judge the impact of the NETI on their ratings. Of the

313 who responded to that question, 15% stated that either their ratings did not change following

the NETI or the workshop had no effect on whatever change may have occurred, 32% said it had

a slight effect, another 32% a moderate effect, and 21% credited the NETI with having a

substantial effect. Given the responses to the previous question, for the overwhelming majority

of the respondents reporting an effect, the effect was clearly positive. Also, inspection of

individual responses shows that many who reported negative or negligible changes in their

ratings had high ratings to begin with, so there was nowhere to go but down.

0

10

20

30

40

50

60

Nu

mb

er

of

res

po

nd

en

ts

1-1

.99

2-2

.49

2.5

-2.9

9

3-3

.49

3.5

-3.9

9

4-4

.19

4.2

-4.3

9

4.4

-4.5

9

4.6

-4.7

9

4.8

-4.8

9

4.9

-5

Average student rating in past 2 years (1-5)

Undergrad (N=287)

Grad (N=207)

0

5

10

15

20

25

30

35

% i

nd

icati

ng

sp

ecif

ied

ch

an

ge

Su

bsta

nti

al

decre

ase

Mo

dera

te

decre

ase

Sli

gh

t

decre

ase

No

ch

an

ge

Sli

gh

t

incre

ase

Mo

dera

te

incre

ase

Su

bsta

nti

al

incre

ase

The pre-post workshop response comparisons tell a similar story. Forty-seven of the 2005

and 2006 workshop participants estimated their student ratings before the workshop and 2–3

years after it, with the results shown in Table 6. Of those 47 respondents, 64% reported

increases, 15% reported no change, and 21% reported decreases, and the increase in the mean

rating was close to statistically significant at the .05 level. Since the percentage of decreased

Figure 1. Student rating histograms

N=290

Figure 2. Changes in student ratings following the NETI.

student ratings was substantially greater than the 6% reported in the Alumni Survey (Figure 2),

we examined how the 10 respondents who reported pre-post workshop ratings decreases

answered Alumni Survey Questions 10 (“What effect do you think the NETI had on your

teaching ratings?”) and 11 (“What effect do you think the NETI had on the quality of your

teaching regardless of ratings?”). Of the nine who responded to both questions, five thought that

their ratings had been improved or unchanged (suggesting that either their pre-workshop or post-

workshop estimates may have been inaccurate), one thought that increases in his teaching load

and changes in his responsibilities accounted for the drop in ratings, and all thought that the

NETI had improved their teaching.

Table 6. Pre-Workshop and Post-Workshop Student Ratings (N=47)

Student ratings*

Pre-workshop mean (SD)

Post-workshop mean (SD)

p**

3.75 (0.69)

3.96 (0.59)

.059

% reporting increases

% reporting no change

% reporting decreases

64%

15%

21%

* 5=excellent, 3=average, 1=poor

** Matched-pair t test

We conclude that increases in student ratings following the NETI were far more common

than unchanged or decreased ratings. The decreases that did occur were slight, and the few

instructors who experienced them were nevertheless likely to view their teaching as having

become more effective as a consequence of their participation in the workshop.

Beliefs about Teaching

. Question 19 of the survey asked the respondents to express their agreement or

disagreement with several teaching strategies and beliefs about teaching. Table 7 lists the

statements in roughly decreasing order of their emphasis in the NETI, states the position

advocated in the NETI, and summarizes the responses. In the discussion that follows, we will

briefly outline the positions the NETI advocates without going into detail on their theoretical and

empirical bases. Interested readers can find justifications in standard references on pedagogy,

including McKeachie & Svinicki.16

Most engineering classes are taught in a traditional lecture format, with PowerPoint being

used to a growing extent. By far the most heavily emphasized message in the NETI is that

students should be actively engaged in classes for meaningful learning to take place, and in

recent years the column “Death by PowerPoint”17

has been included in the NETI notebook to

support several warnings about over-reliance on prepared visuals in lectures. The survey

respondents heavily supported the point of view advocated in the NETI: 73% agreed that class

activities are desirable and only 3% disagreed, and less than 4% agreed that lectures should be

delivered primarily via prepared visuals and 79% disagreed.

Table 7. Beliefs about Teaching

Belief

NE

TI

po

sit

ion

Ro

le

in N

ET

IA

Attitude

B

Pre-Test Response Mean (SD)

C

Post-Test Response Mean (SD)

C

pS

Teachers should give students frequent course-related

group activities in class. Yes (a)

(1)–0%, (2)–3%, (3)–24%,

(4)–51%, (5)–22% 3.84 (0.76) 3.98 (0.79) .27

Teachers should deliver lectures primarily via

transparencies or PowerPoint. No (a)

(1)–40%, (2)–39%, (3)–18%,

(4)–3%, (5)– <1% 2.00 (0.92) 1.76 (0.84) .03*

Giving tests that only the top students have time to

complete sorts students effectively according to their

understanding of the material.

No (b) (1)–34%, (2)–46%, (3)–14%,

(4)–6%, (5)–0% 2.39 (0.83) 1.88 (0.82) .0007***

Teachers should give students detailed study guides

for tests. Yes (b)

(1)–4%, (2)–19%, (3)–42%,

(4)–27%, (5)–8% 2.98 (0.84) 3.53 (0.92) .003**

You can teach people to be critical thinkers. Yes (b) (1)–1%, (2)–2%, (3)–13%,

(4)–63%, (5)–21% 3.90 (0.73) 3.96 (0.92) .58

You can teach people to be creative thinkers. Yes (b) (1)–1%, (2)–7%, (3)–25%,

(4)–54%, (5)–13% 3.61 (0.88) 3.63 (0.87) .91

The most important characteristic of good teaching is to

have complete and accurate lecture notes. No (c)

(1)–13%, (2)–29%, (3)–35%,

(4)–19%, (5)–4% 2.67 (0.95) 2.80 (0.94) .33

It is acceptable for teachers to give assignments on

material that has not been explicitly covered in lectures.—

(c)

(1)–9%, (2)–48%, (3)–24%,

(4)–14%, (5)–5% 3.57 (0.85) 3.33 (1.01) .10

Legend

A. (a)–major topic, (b)–moderately addressed, (c)–slightly addressed

B. N=308. (1)–strongly disagree, (2)–disagree, (3)–neutral, (4)–agree, (5)–strongly agree

C. N=51. (1)–strongly disagree, (2)–disagree, (3)–neutral, (4)–agree, (5)–strongly agree

D. Significance level from Wilcoxon’s Matched Pairs Test: *=significant at .05 level, **=significant at .01 level, ***=significant at .001 level

Since a substantial fraction of the participants indicated that they used activities in their

teaching and that they had been motivated to do so by the NETI (Table 5), it appears that the

NETI was successful in promoting a positive attitude toward active learning. The gain in

awareness and use of active learning shown by the pre-post test results was not significant,

perhaps reflecting a growing awareness of active learning among younger faculty members,

whose mean pre-test score was quite high. By the same token, the growing practice of delivering

lectures primarily using prepared visuals did not have many supporters, and the pre-post test

results indicate that the NETI played a significant role in raising the respondents’ level of

disapproval.

A common feature of traditional engineering courses is a heavy reliance on long

quantitative problem-solving tests that few students can finish, with average grades commonly

on the order of 60/100 or less. A heavily stressed point in the NETI is that such tests discriminate

unfairly against students who work carefully and methodically but slowly, who have the

potential to be excellent engineers but are likely to fail long tests even if they understand the

material. In their survey responses, 80% of the respondents disagreed with the statement that

long tests sort students according to how well they understand the material and only 6% agreed

with it. The level of approval dropped significantly in the pre-post test results.

Another common characteristic of engineering tests is that they often contain surprises,

such as problems requiring mathematical manipulations different from any that the students have

practiced in homework, with the rationale being that we need to find out if the students can

“think for themselves.” Like the ability to solve quantitative problems quickly, the ability to

figure out such puzzles in the artificial environment of a time-limited test has never been shown

to correlate with success in engineering or science; in fact, several studies have shown an almost

complete lack of correlation between grades in college and subsequent career success.18–20

In the

NETI, we propose that the more instructors make their expectations explicit and clear—

particularly expectations of high-level thinking and problem-solving skills—the more likely their

students will be to meet those expectations.

A strong NETI recommendation (particularly in the past 5–10 years) is for instructors to

express their expectations as learning objectives and share them with the students in study guides

for tests. This is one of our most controversial recommendations, since it contradicts a common

faculty belief that making expectations too explicit amounts to “spoon-feeding” or “coddling”

the students, lowering standards and opening the door to unqualified students being certified to

practice engineering. Only 35% of the respondents agreed with the recommendation but even

fewer (23%) disagreed with it, and there were more neutral responses (42%) than for any other

issue. The pre-post test results are much more encouraging, showing a highly significant shift

from neutrality to approval.

The next two questions had to do with whether critical thinking and creative thinking can

be taught. Many people think that they cannot be, particularly creativity, which they regard as an

innate talent that one either has or not. The NETI does not focus on how to teach those two

skills, but it strongly recommends that instructors write learning objectives that address Levels 5

(evaluating, or critical thinking) and 6 (creating, or creative thinking) on Bloom’s Taxonomy and

then provide students with practice and feedback on tasks that require thinking at those levels.

Eighty-four percent of the survey respondents agreed that critical thinking can be taught (3%

disagreed) and 67% agreed that creative thinking can be taught (8% disagreed). The mean pre-

workshop level of agreement was fairly high among the 2005 and 2006 respondents, and was not

materially changed by participating in the NETI.

Next, when presented with a statement that the most important characteristic of good

teaching is having complete and accurate lecture notes, 23% agreed and 42% disagreed, with the

approval rate increasing slightly in the pre-post test results. This statement expresses the

common misconception of many new faculty members that if they prepare good lecture notes,

effective teaching and learning will automatically follow.10

We speculate that the level of

agreement would have been even lower if the question had been better worded, since some

respondents might have equated disagreement with a belief that complete and accurate notes are

not important.

The final statement (“It is acceptable for teachers to give assignments on material that

has not been explicitly covered in lectures.”) is problematic. Our intention was to ask about

beginning instruction with challenges and then presenting material on a need-to-know basis in

the context of the challenges (as inductive teaching methods such as inquiry-based and problem-

based learning do), but the statement sounded more like a proposal to grade students on

assignments for which they have not been prepared, which is antithetical to the NETI

philosophy. Not surprisingly, only 19% of the respondents agreed with the statement and 57%

disagreed, and the approval rate decreased slightly in the pre-post test results. We attribute this

outcome both to the poor wording of the question and the peripheral role of inductive methods in

the workshop.

Involvement in Instructional Development

One of the goals of the NETI founders was to encourage and equip participants to give

presentations to colleagues on what they learned at the NETI. Question 12 of the survey asked

about the participants’ involvement with instructional development. Of the 313 respondents to

this question, 44% had been involved (9% extensively and 35% occasionally), 21% had not been

involved but planned to be in the future, 32% had no prior or anticipated involvement, and 3%

responded with “other.” In short, 65% of the respondents were interested in instructional

development, the majority of whom had actually done it. Question 13 then asked “Did the NETI

motivate you to get involved in instructional development?” Of the 313 respondents, 52%

responded affirmatively and 48% negatively, with the latter including some individuals who had

begun doing instructional development before attending the NETI. The clear inference is that the

NETI succeeded in transforming some of its participants into potential change agents on their

campuses.

Engagement in Scholarly Teaching and the Scholarship of Teaching and Learning

Hutchings and Shulman4 introduced the now familiar distinction between scholarly

teaching and the scholarship of teaching and learning. Instructors doing scholarly teaching

inform themselves of the latest ideas in pedagogy and assessment, take those ideas into account

as they attempt to improve their teaching, and subject their attempts to self- and/or external

assessment and evaluation. Instructors engaged in the scholarship of teaching and learning

approach teaching in the same scholarly way, but in addition they present and publish

descriptions of their teaching innovations and their assessment results in a form others can

evaluate, replicate, and build on.

Question 18 of the survey asked about the participants’ engagement in scholarly

teaching—as evidenced by their belonging to ASEE, attending education conferences, and

reading education-related journals—and whether attending the NETI motivated their

engagement. Question 16 asked about their involvement in doing unpublished classroom

research (further evidence of scholarly teaching) and in the scholarship of teaching and

learning—presenting research results at a conference and publishing results of unfunded and

funded research. Question 17 asked whether the NETI had motivated them to do educational

research. Responses are summarized in Table 8.

Table 8. Engagement in Scholarly Teaching and Educational Scholarship (N=309)

Yes

NETI

Motivateda

bRead education-related journal articles? 89% 47%

bParticipated in an education conference 73% 31%

bBelong to ASEE 69% 21%

bDone educational research of any type 76% 50%

bDone unpresented & unpublished classroom research 25%

cPresented educational research results at a conference 33%

cDone unfunded educational research and published results 27%

cDone funded educational research and published results 25%

aPercentages are of the total number of respondents (309)

bScholarly teaching

cScholarship of teaching and learning

Substantial percentages of the survey respondents reported engaging in practices

associated with scholarly teaching, with many indicating that the NETI played a role in

motivating them to do so. Close to 90% had read education-related journal articles (47% were

motivated by NETI), over 70% participated in at least one education conference (31% NETI);

around 70% belonged to ASEE (21% NETI); and 25% had done unpublished classroom

research. A concept related to scholarly teaching is reflective practice,7 wherein teachers reflect

on and attempt to understand the general processes of teaching and learning and as much as they

can about how their students function and what facilitates their learning. The open-ended

responses shown in Table 4 make it clear that the NETI served to make many of the respondents

more reflective practitioners and so more scholarly teachers.

The NETI devoted relatively little time to promoting the scholarship of teaching and

learning, and the fact that it had that effect on so many participants came as a pleasant surprise.

Three-quarters of the respondents reported having done informal classroom research and/or

formal educational research, with 50% indicating that their research activity was motivated by

the NETI. A third of the respondents had reported on their research at a conference, and roughly

a quarter of them had published research results. The survey did not ask the respondents to

indicate the role of the workshop in promoting each individual category of research involvement,

but the other responses in Table 8 suggest that significant fractions of those who have presented

or published research were stimulated to do so by the NETI.

Recollections and Comments

Question 20 invited the respondents to share their recollections of the NETI and to offer

any comments they might have about it. There were 208 responses to this invitation. Most of the

recollections (189) were clearly positive, ranging from “It was good” to detailed descriptions of

major career changes induced by the NETI. Several examples follow.

NETI expanded, deepened, and changed what I thought I knew about learning

and teaching and forced me to think about every aspect of my course preparation.

The workshop helped me to focus on seeing my teaching from the student’s

perspective and to aim to optimize student learning.

Attended about ten years ago. Two specific things I did as a result: 1) I started

keeping a box of Kleenex in my desk (and have been twice nominated for the

university-wide advising award); 2) I give many more open book exams. I now do

many more group activities and problem sessions. It slows [me] down to the

students’ pace and, if they are willing, it engages them. The biggest impact was

just getting a sense of how hard and honestly good teachers work at teaching.

A few respondents said that they liked the workshop and had tried some of the ideas from the

NETI but had been discouraged by resistance from students and/or faculty colleagues and/or a

lack of support for teaching in the faculty reward system, and they subsequently went back to

more traditional teaching:

When I first got back I tried several of the techniques over the course of a couple

of years with mixed results. I still retain a few of them, but students are generally

reluctant to embrace them, at least from me.

I enjoyed NETI and it opened my eyes to what academia should or could be.

Unfortunately, with a dictator department head and lack of support from senior

faculty, I felt hampered and discouraged from many activities.

Some who had taken the NETI as relatively new teachers talked about how it had gotten their

careers off to a good start:

I enjoyed NETI, and was grateful that my dean identified this opportunity for me

as a new faculty member. It reinforced some of the “crazy” ways I was running

my classroom compared to the older faculty, and it gave me new ideas about why

my teaching methods weren’t working for all my students prior to NETI.

Others who were more experienced when they attended noted that the NETI helped them to

clarify their ideas about teaching and to better understand some of the techniques they had

discovered independently.

Years ago I developed my own teaching style by emulating those who I thought

were excellent teachers, i.e., those teachers whose style worked for me. The NETI

helped me to understand why most of the stuff I do works but it also opened my

eyes to several new things [learning styles, learning objectives, group work]

which I am convinced have improved my teaching.

The main effect of NETI was to reinforce what I already knew and to demonstrate

the principles. It was important that I was a participant in the active learning

techniques so that I could experience what my students experience. NETI also

helped me to think about ways to use active learning in my conference

presentations.

Some mentioned using the NETI as an inspiration or model for their own faculty development

activities.

NETI was one of my great experiences as a faculty member. It motivated me not

only to introduce a repertoire of learning activities in class, but also to increase

my involvement as a leader of teaching workshops.

I came to NETI to prepare (i.e., steal ideas) for a teaching workshop I was

planning. It was very successful in giving me a solid overview of topics I should

include. Most importantly, it confirmed my suspicion that I should not focus the

workshop on teaching specific [disciplinary] topics but on broader learning

methods, such as active learning. The resulting workshops were highly successful.

Several spoke in terms of the NETI having a significant impact on their career beyond simply

improving their teaching.

I still remember NETI vividly, and I think of the workshop as a defining moment

in my career. The workshop had a profound impact on my enthusiasm for

teaching. After it, I was really sure that I wanted to devote my career to

engineering education and that I would find a way to craft a career with a main

focus in this area. I am fortunate to work with colleagues who inspire, mentor,

support, and encourage me to reach my potential as an engineering educator.

NETI was a “professional life-changing” experience for me. I was struggling with

all sorts of issues making the transition from being a traditional researcher in

BioChE into doing something more “educational.” At many levels [the NETI]

transformed my approach not just to classroom teaching, but to professional life

in general. It confirmed many of the things I was doing in the classroom, but gave

me so much more to use, and helped me change what needed changing. It

catalyzed a path to educational scholarship in general that probably would not

have happened otherwise.

And finally, a number of respondents were struck by the communal experience provided by the

NETI.

What I do remember of NETI is that I really enjoyed it, talking with others,

getting exposed to new ideas, revisiting existing ideas in new ways. Perhaps what

NETI did as much as anything was help me keep my enthusiasm up, remind me

that there are lots of educators who want to talk about teaching and engage in

thinking deeply about it.

Of the 208 responses to Question 20, 15 could be characterized as neutral, expressing

neither satisfaction nor dissatisfaction with the NETI. Most of these respondents said something

to the effect that they couldn’t remember much about the workshop (“I only recollect that it was

in conjunction with an ASEE meeting in Anaheim, a long time ago.”); others made remarks about

the workshop having no lasting effects (“There was an immediate short-term impact but, over

time, the impact diminished.”); and several made noncommittal remarks such as “It was OK.”

Only four respondents were distinctly dissatisfied. One thought that there was too much

lecturing in the workshop and also expressed doubts about the value of teaching workshops in an

educational climate that does not support quality education. Another complained about

insufficient emphasis in the workshop on the basics of classroom delivery. The third found some

of the information too “preachy” and did not find any applications that he/she could take into the

classroom, and the fourth expressed frustration over the negative impact of the NETI on his/her

research productivity:

Effective teaching, and getting motivated to try new things as the result of NETI,

actually created some potential problems for me. (I did get tenure and promotion,

on time, by the way. I have taken a little longer than most to reach full prof., but it

has happened.)

SUMMARY

We are using multiple measures as indicators of the impact of the NETI on its

participants’ teaching and by inference on their students’ learning. At the lowest level of

assessment, we have summarized the participants’ satisfaction with the workshop, the rationale

being that participants who are unhappy with a teaching workshop are unlikely to incorporate

workshop recommendations into their teaching practices. We then examined the extent to which

the NETI helped make participants aware of proven but nontraditional teaching methods and

influenced them to adopt those methods; influenced their attitudes and beliefs about teaching and

learning; affected how their students felt about the quality of their teaching; engaged them in

instructional development on their home campuses and beyond; inspired them to be more

scholarly and reflective in their teaching; and motivated them to engage in educational research.

The outcomes are summarized below.

• Participants’ satisfaction with the NETI. The level of immediate post-workshop

participant satisfaction with the NETI has been extremely high. In its 18 offerings, the

workshop has received a total of 820 ratings of which all but five were positive (649

“Excellent” and 122 “Good”) and the remaining five were “Average”. The retrospective

evaluations gathered in the Alumni Survey were similarly affirming. Of 165 open-ended

assessments of the effect of the workshop on teaching quality, 87% were clearly positive,

13% were neutral, and none were clearly negative, and of 208 shared recollections of the

workshop, 91% were positive, 7% were neutral, and 2% were negative. We conclude that the

level of participant satisfaction with the workshop is as high as anyone could wish for.

• Awareness and use of effective teaching practices. The teaching strategies most heavily

emphasized in the NETI are designing instruction to address the full spectrum of student

learning styles, writing comprehensive learning objectives and using them as the basis for

course planning, instruction, and assessment, and getting students actively engaged in course-

related tasks during classes (active learning). As Table 5 shows, substantial percentages of

the respondents incorporated learning styles, learning objectives, and active learning in their

teaching and credited the NETI with having a moderate or strong influence on their doing so.

These results along with many responses reporting adoptions of other NETI-recommended

strategies make it clear that the workshop had a significant impact on participants’ teaching

practices, and since the three named strategies have been repeatedly shown to facilitate

learning, it is fair to infer that the teaching changes made by the NETI participants led to

improved learning by their students.

• Attitudes toward teaching, learning, and students. The NETI clearly made its participants

more student-centered in their teaching. For example, 73% of the survey respondents agreed

that teachers should give frequent group activities in class and only 3% disagreed; awareness

and use of other student-centered teaching methods such as cooperative learning, problem-

based learning, and inquiry-based learning was also quite high; the gains in awareness and

use of all of these methods from before the workshop to after it were statistically significant;

and many respondents credited the NETI with motivating them to use the methods. Also,

many of the free responses in Table 4 expressed an improved understanding of and/or rapport

with students.

• Student ratings. Of 290 respondents, 67% reported increased student ratings following the

NETI, 29% saw no change (some of whom had close to the maximum rating before they

came and so there was little room for improvement), and 6% experienced decreased ratings,

with only one of the decreases being substantial. In the pre-post workshop test results, the

mean rating increased to an extent slightly short of the .05 level of significance, with 64%

reporting increases, 15% no change, and 21% decreases, most of them slight. In their open-

ended responses, many of those who experienced drops in their student ratings expressed

beliefs that the NETI-motivated changes in their teaching had improved their students’

learning.

• Instructional development. Fifty-two percent of 313 respondents felt that the NETI

motivated them to get involved in instructional development; 44% had engaged in it (9%

extensively and 35% occasionally), and 21% had not been involved but planned to engage in

it in the future. The goal of the NETI to convert at least some of its participants into change

agents on their home campuses was clearly realized.

• Scholarly teaching and the scholarship of teaching and learning. Significant percentages

of the respondents reported engaging in practices that by definition characterize scholarly

teaching: 89% stated that they read education-related journal articles and 73% had

participated in an education conference, with roughly half of each group having been

motivated to do so by the NETI; and 69% belonged to the ASEE, roughly a third of whom

were persuaded by the NETI to join. Three-quarters of the respondents had engaged in

classroom research and/or formal educational research, with 50% having been stimulated to

do so by the NETI.

In short, the Alumni Survey results strongly suggest that the NETI successfully motivated

many of its participants to adopt or increase their use of proven teaching strategies known to

correlate with improved student learning; made them more student-centered, scholarly, and

reflective in their teaching practice; increased the student ratings of most of them and decreased

the ratings of very few; and induced a number of them to engage in instructional development

and educational scholarship.

DISCUSSION AND RECOMMENDATIONS

What makes the NETI as successful as it has been?

Engineering faculty members are not noted for their enthusiasm about teaching

workshops. Directors and staff of campus teaching and learning centers frequently complain that

few engineers attend their workshops, and those who come tend to dismiss what they hear as

irrelevant to their courses, students, and problems. There are significant exceptions to this

pattern, however: some well-established instructional development programs have attracted and

influenced many engineering faculty members. As the previous section indicates, the National

Effective Teaching Institute falls in this category.

The question is, what do the successful programs do that most programs fail to do? We

propose that the answer lies in the adult learning literature. Wlodkowski21

suggests that five

attributes of a learning environment have a motivational effect on adult learners (see Table 9).

Let us first consider the expertise of the presenters (more precisely, the workshop

participants’ perception of the presenters’ expertise). Teaching and learning center directors and

staff tend to come from education and psychology, and most workshops they give are intended to

address faculty in all disciplines. The presenters are normally quite knowledgeable about

learning theories and good pedagogical practices; however, they lack the disciplinary content

knowledge to construct examples that would make the workshop material clearly relevant to

engineering faculty, and even if they had that knowledge they would probably refrain from using

it for fear of losing participants from other disciplines. Some engineers may be perceptive

enough to see immediately how they could apply the workshop content in their courses but most

are not, and those who don’t see the relevance of the content tend to reject the idea that the

presenters can tell them anything useful about how to teach.

Table 9. Factors that Motivate Adult Learning19

1. Expertise of presenters. Adults expect their teachers/trainers to be experts in the material they are presenting and to be well-prepared to teach it. While teachers may occasionally be able to "wing it” successfully, adults are usually quick to detect inadequate background knowledge and/or lack of systematic preparation.

2. Relevance of content. Adults may quickly become impatient with material for which they cannot see an immediate use. Once they see the relevance of workshop content to their needs and interests, they will be much more willing to attend to presentations of general instructional methodology and theory.

3. Choice in application. Adults respond particularly well when encouraged to be flexible in choosing how and when to apply recommended methods. They are skeptical (usually appropriately) of “one size fits all” prescriptions, having learned by experience that no two situations are ever exactly alike.

4. Praxis (action plus reflection). Adults appreciate opportunities to see implementations of methods being taught, to try the methods themselves, and to reflect on the outcomes.

5. Groupwork. Extensive research on active and cooperative learning has shown conclusively that working in groups enhances learning and helps participants form learning networks.

13,14

While some workshop participants (like many students) may initially feel skeptical about group activities, as long as the activities are well implemented and clearly relevant to the workshop learning objectives, most participants eventually come to appreciate them.

Many campus teaching workshops also fail to meet the other criteria listed in Table 9. An

overemphasis on general educational theories and the absence of engineering-specific examples

that cause engineers to discount the expertise of the presenters (Criterion 1) also cause them to

consider the workshop content irrelevant to their needs and interests (Criterion 2). Teaching

workshops are frequently prescriptive in their recommendations (to teach well, you must do x, y,

and z), giving the participants no choice in whether, when, and how to implement each

recommendation (Criterion 3). The participants get the message that they have been teaching

wrong and must make the recommended changes to be acceptable teachers, a message most

don’t appreciate. Many workshops consist almost entirely of lectures on educational theories and

methods, with no demonstrations or opportunities to practice the methods (Criterion 4), and

provide few or no opportunities for interactions among participants (Criterion 5). In an absurd

extreme of a Criterion 5 violation, one of the authors once attended a workshop on active

learning that consisted of three hours of straight lecturing on the importance of getting students

active in class!

We believe that the success of the NETI derives in large measure from the extent to

which it has satisfied Wlodkowski’s criteria in addressing its audience of engineering educators.

1. Expertise. At least two of the facilitators in each NETI offering and all of the NETI

fellows have been award-winning engineering educators. In addition, the presenters have

all made substantive contributions to the engineering education literature, and much of

their work is highlighted in the workshop presentation. Whatever else the participants

may think, they cannot claim that the presenters don’t understand engineering content,

students, and classroom problems.

2. Relevance. Engineering-specific examples are used throughout the workshop. When an

example is given of how to teach a course topic in a way that addresses the full spectrum

of learning styles, the course is fluid dynamics; when illustrative learning objectives are

shown and critiqued by the participants, they are objectives for common engineering

courses; when videoclips are shown to illustrate how active learning can be implemented

in a large lecture class, the clips are of one of the workshop presenters teaching an

engineering class.

3. Choice in application. The message is repeatedly given in the workshop that if the

participants attempt to implement every recommendation in the first week of their next

course, they will fail so thoroughly that they will never want to try anything new again.

They are cautioned instead to choose just a few new ideas that look reasonable; try them

several times to get a sense of how well they work; keep doing the ones that work and

drop those that don’t; and add another one or two ideas in the next semester. They are

also assured repeatedly that there are no recipes for successful teaching. They will need

to experiment to find the balances that work best for them between theoretical and

practical course content, lecturing and active learning, individual and group work, etc.

4. Praxis. Almost nothing is taught in the NETI that is not illustrated by demonstrations

and/or participant activities. After different learning styles are described, the participants

assess their own styles and reflect on how those styles are different from the styles of

most of their students. They critique good and bad learning objectives for engineering

courses and then write their own; they critique a poorly written test and contrast it with a

much better but still imperfect version of the same test; they generate and critique

possible instructor responses to common problem scenarios (disruptive classes, students

in crisis, cheating,...), and so on. Over the years, many participants have remarked that

they appreciate the extent to which we practice what we preach.

5. Groupwork. The workshop facilitators rarely present for more than 15–20 minutes

without engaging the participants in content-related activities. The activities are

sometimes done individually, sometimes in groups of 2–4, and sometimes individually

followed by small group processing (“think-pair-share”). Even with that level of activity,

a common request in post-workshop evaluations is for more.

Recommendations for instructional development in engineering

Based on the results of the NETI Alumni Survey and the discussion just concluded, we

offer the following recommendations for making teaching workshops for engineering faculty

(and by extension, for engineering graduate teaching assistants and graduate students

participating in “Preparing the Professoriate” programs) as effective as possible.

1. Design workshops specifically for engineering faculty and perhaps members of related

disciplines (particularly physical and mathematical sciences). In trying to reach everyone

with a teaching workshop, you run a considerable risk of reaching no one.

2. Bring in engineering faculty members who are excellent teachers to present the

workshops or to co-present them with experts in general pedagogy.

3. Illustrate most or all recommended teaching methods with examples and demonstrations

drawn from engineering courses. To the greatest extent possible, use the recommended

methods yourself in presenting the workshop (i.e., practice what you preach).

4. Suggest, don’t prescribe; give choices; and caution participants not to try too much new

at once.

5. Get the participants to work in small groups to generate many of the ideas you want to

present to them. If you just list the ideas, you’re lecturing at them; if the participants

generate the ideas themselves, you’re affirming them. When you use the latter approach,

the participants are much more likely to be receptive to the ideas and appreciative of you

for acknowledging and empowering them.

REFERENCES

1. Van Note Chism, N., & Szabó, B.S. (1997). How faculty development programs evaluate their services. J. Staff,

Prog., & Org. Dev., 15(2), 55–62.

2. D’Eon, M., Sadownik, L., Harrison, A., & Nation, J. (2008). Using self-assessments to detect workshop

success: Do they work? Am. J. Evaluation, 29(1), 92–98.

3. Boyer, E. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: Carnegie Foundation

for the Advancement of Teaching.

4. Hutchings, P., & Shulman, L.S. (1999). The scholarship of teaching: New elaborations, new developments.

Change, 31(5), 10–15.

5. Sue, V.M., & Ritter, L.A. (2007). Conducting online surveys. Thousand Oaks, CA: Sage, 2007.

6. Babbie, E. (2004). The practice of social research (10th ed.). Belmont, CA: Wadsworth Publishing Co.

7. Schön, D.A. (1983). The reflective practitioner. New York: Basic Books.

8. Stice, J.E. (1976). A first step toward improved teaching. Engr. Education, 66(5), 394–398.

9. Lochhead, J. and Whimbey, A. (1987). Teaching analytical reasoning through thinking aloud pair problem

solving, in J.E. Stice, ed., Developing critical thinking and problem-solving abilities, New Directions for

Teaching and Learning, No. 30, pp. 73–92. San Francisco: Jossey-Bass.

10. Boice, R. (1992). The new faculty member. San Francisco: Jossey-Bass.

11. Felder, R.M., & Brent, R. (2005). Understanding student differences. J. Engr. Education, 94(1), 57–72.

<www.ncsu.edu/felder–public/Papers/Understanding_Differences.pdf>.

12. Hartley, J., & Davies, I.K. (1976). Preinstructional strategies: The role of pretests, behavioral objectives,

overviews, and advance organizers. Rev. Educ. Res., 46(2), 239–265.

13. Prince, M. (2004). Does active learning work? A review of the research. J. Engr. Education, 93(3), 223–231.

14. Johnson, D.W., Johnson, R.T., & Stanne, M.E. (2000). Cooperative learning methods: A meta–analysis.

Minneapolis: University of Minnesota Cooperative Learning Center.

<http://www.co–operation.org/pages/cl–methods.html>.

15. Felder, R.M., & Brent, R. (2008). Student ratings of teaching: Myths, facts, and good practices. Chem. Engr.

Education, 42(1), 33–34. <www.ncsu.edu/felder–public/Columns/StudentRatings.pdf>.

16. McKeachie, W.J., & Svinicki, M. (2005). McKeachie’s teaching tips: Strategies, research, and theory for

college and university teachers (12th ed.). Florence, KY: Cengage Learning.

17. Felder, R.M., & Brent, R. (2005). Death by PowerPoint. Chem. Engr. Education, 39(1), 28–29.

<www.ncsu.edu/felder–public/Columns/PowerPoint.pdf>.

18. Stice, J.E. (1979). Grades and test scores: Do they predict adult achievement? Engr. Education, 69(5), 390–393.

19. Cohen, P.A. (1984). College grades and adult achievement: A research synthesis. Res. in Higher Ed., 20(3),

281–293.

20. Samson, G.E., Graue, M.E., Weinstein, T., & Walberg, H.J. (1984). Academic and occupational performance:

A quantitative synthesis. Am. Educ. Res. Journal, 21(2), 311–321.

21. Wlodkowski, R.J. (1999). Enhancing adult motivation to learn: A guide to improving instruction and increasing

learner achievement. San Francisco: Jossey–Bass.

Appendix A

NETI Participants’ Institutions — 1991-2008

1. Ajou Univ. (Korea) (2)

2. Arizona State Univ. (5)

3. Auburn Univ. (1)

4. Baylor Univ. (2)

5. Binghamton Univ. (4)

6. Boise State Univ. (3)

7. Brigham Young Univ. (1)

8. Bucknell Univ. (12)

9. California State Univ. – Los Angeles (4)

10. Case Western Reserve Univ. (5)

11. Central Connecticut State Univ. (2)

12. Clarkson Univ. (1)

13. Clemson Univ. (25)

14. Cleveland State Univ. (2)

15. Colorado School of Mines (2)

16. Colorado State Univ. (2)

17. Cornell Univ. (10)

18. Cuyahoga Community College (1)

19. Dartmouth College (4)

20. DeVry Inst. Of Technology – Addison (1)

21. DeVry Inst. Of Technology – Columbus (1)

22. Duke Univ. (3)

23. Ecole Polytechnique de Montreal (4)

24. Ecole de Technologie Superieure (Montreal) (1)

25. Embry-Riddle Aeronautical Univ. (3)

26. Florida Gulf Coast Univ. (2)

27. Florida Inst. Of Technology (1)

28. Florida International Univ. (3)

29. Franklin Univ. (1)

30. Gaston College (1)

31. Georgia Inst. Of Technology (9)

32. George Washington Univ. (1)

33. Gonzaga Univ. (3)

34. Grand Valley State Univ. (1)

35. Howard Univ. (3)

36. Humboldt State Univ. (1)

37. Illinois Inst. Of Technology (4)

38. Indiana Univ.-Purdue Univ. at Indianapolis (15)

39. Iowa State Univ. (11)

40. ITESM-CEM (Mexico) (1)

41. Kansas State Univ. (10)

42. Kettering Univ. (12)

43. Kuwait Univ. (1)

44. Lafayette College (2)

45. Lake Superior State Univ. (2)

46. Lamar Univ. (2)

47. Lawrence Technological Univ. (1)

48. Lehigh Univ. (6)

49. Louisiana State Univ. (1)

50. Louisiana Tech Univ. (7)

51. Loyola Marymount Univ. (4)

52. Manhattan College (3)

53. Marquette Univ. (2)

54. Massachusetts Inst. Of Technology (1)

55. Mercer Univ. (3)

56. Metropolitan State College of Denver (1)

57. Miami Univ. (Ohio) (4)

58. Michigan State Univ. (13)

59. Michigan Technological Univ. (8)

60. Milwaukee School of Engineering (7)

61. Mississippi State Univ. (10)

62. Missouri Univ. of Science and Technology (36)

63. Montana State Univ. (3)

64. New Jersey Inst. Of Technology (2)

65. New Mexico Inst. Of Mining and Technology (3)

66. North Carolina A&T State Univ. (8)

67. North Carolina State Univ. (2)

68. North Dakota State Univ. (6)

69. Northeastern Univ. (6)

70. Northern Alberta Inst. Of Technology (2)

71. Northern Arizona Univ. (2)

72. Northern Illinois Univ. (3)

73. Northwestern Univ. (2)

74. Ohio Northern Univ. (3)

75. Ohio State Univ. (4)

76. Ohio Univ. (10)

77. Oklahoma Christian Univ. (2)

78. Oklahoma State Univ. (2)

79. Old Dominion Univ. (1)

80. Olivet Nazarene Univ. (1)

81. Oregon State Univ. (1)

82. Penn State Univ. (1)

83. Penn State Univ. – Erie (1)

84. Polytechnic Inst. Of New York (2)

85. Portland State Univ. (1)

86. Purdue Univ. (18)

87. Purdue Univ. – Calumet (1)

88. Purdue Univ. – Columbus (1)

89. Queens Univ. (1)

90. Rensselaer Polytechnic Inst. (2)

91. Rice Univ. (3)

92. Rochester Inst. Of Technology (14)

93. Rose-Hulman Inst. Of Technology (8)

94. Rowan Univ. (13)

95. Royal Military College of Canada (2)

96. Ryerson Univ. (1)

97. Saginaw Valley State Univ. (2)

98. San Jose State Univ. (1)

99. Santa Clara Univ. (2)

100. Seattle Univ. (1)

101. South Dakota State Univ. (14)

102. Southern College of Technology (1)

103. Southern Illinois Univ. (9)

104. Stanford Univ. (1)

105. Stevens Institute of Technology (2)

106. St. Louis University (1)

107. SUNY – Binghamton (1)

108. SUNY – Buffalo (7)

109. Texas A&M Univ. (11)

110. Texas A&M Univ. – Qatar (3)

111. Texas Tech Univ. (3)

112. The Citadel (2)

113. Trinity Univ. (2)

114. Tri – State Univ. (10)

115. Tufts Univ. (1)

116. Tulane Univ. (4)

117. Tuskegee Univ. (2)

118. Univ. De Las Americas – Puebla (Mexico) (2)

119. Univ. Del Turabo (Mexico) (1))

120. Univ. Del Valle (Colombia) (1)

121. Univ. Of Akron (11)

122. Univ. Of Alabama (7)

123. Univ. Of Alberta (10)

124. Univ. Of Ancona (Italy) (1)

125. Univ. Of Arizona (13)

126. Univ. of Arkansas (4)

127. Univ. of Calgary (2)

128. Univ. of California – Berkeley (6)

129. Univ. of Colorado (3)

130. Univ. of Colorado at Denver (1)

131. Univ. of Connecticut (6)

132. Univ. of Dayton (2)

133. Univ. of Delaware (2)

134. Univ. of Denver (1)

135. Univ. of Detroit–Mercy (1)

136. Univ. of Evansville (1)

137. Univ. of Florida (13)

138. Univ. of Georgia (2)

139. Univ. of Hartford (1)

140. Univ. of Hawaii (2)

141. Univ. of Houston (2)

142. Univ. of Illinois (5)

143. Univ. of Iowa (1)

144. Univ. of Kansas (12)

145. Univ. of Kentucky (2)

146. Univ. of Louisville (8)

147. Univ. of Maine (7)

148. Univ. of Manitoba (1)

149. Univ. of Massachusetts – Dartmouth (1)

150. Univ. of Memphis (5)

151. Univ. of Michigan (15)

152. Univ. of Michigan – Dearborn (1)

153. Univ. of the Minho (Portugal) (1)

154. Univ. of Missouri – Columbia (1)

155. Univ. of Nebraska (5)

156. Univ. of New Haven (1)

157. Univ. of New Mexico (5)

158. Univ. of North Carolina at Charlotte (9)

159. Univ. of North Dakota (2)

160. Univ. of Notre Dame (7)

161. Univ. of Oklahoma (3)

162. Univ. of the Pacific (5)

163. Univ. of Pittsburgh (2)

164. Univ. of Pittsburgh at Johnstown

165. Univ. of Portland (2)

166. Univ. of Puerto Rico at Mayaguez (1)

167. Univ. of Rochester (1)

168. Univ. of San Diego (4)

169. Univ. of Saskatchewan (2)

170. Université de Sherbrooke (1)

171. Univ. of South Carolina (5)

172. Univ. of Southern Colorado (2)

173. Univ. of South Florida (2)

174. Univ. of Tennessee (4)

175. Univ. of Tennessee at Martin (9)

176. Univ. of Texas – Austin (16)

177. Univ. of Texas – Tyler (4)

178. Univ. of Toledo (3)

179. Univ. of Toronto (3)

180. Univ. of Vermont (1)

181. Univ. of Washington (16)

182. Univ. of Waterloo (3)

183. Univ. of Wisconsin (2)

184. Univ. of Wisconsin–Platteville (5)

185. Univ. of Wyoming (3)

186. U.S. Air Force Academy (14)

187. U.S. Military Academy (4)

188. U.S. Naval Academy (1)

189. Utah State Univ. (5)

190. Valparaiso Univ. (14)

191. Vanderbilt Univ. (2)

192. Villanova Univ. (2)

193. Virginia Military Inst. (2)

194. Virginia Polytechnic Inst. (23)

195. Washington State Univ. (7)

196. Wayne State Univ. (9)

197. Webb Inst. (2)

198. Western Kentucky Univ. (3)

199. Western Michigan Univ. (1)

200. Western Washington Univ. (2)

201. West Virginia Inst. Of Technology (2)

202. West Virginia Univ. (6)

203. Wichita State Univ. (23)

204. Widener Univ. (1)

205. Wilkes Univ. (2)

206. Winona State Univ. (2)

207. Worcester Polytechnic Inst. (2)

208. Wright State Univ. (9)

209. Youngstown State Univ. (1)

Appendix B. NETI Alumni Survey

1. How many years have you taught?

___ <1 ___ 1–2 ___ >2–5 ___ >5–10 ___ >10

2. How many years have you worked in industry, not counting co-op or summer internships when you

were a student?

___ 0 ___ <1 ___ 1–2 ___ >2–5 ___ >5–10 ___ >10

3. How is your current institution classified?

__ RU/VH (Research I university)

__ 4-year university or college, Ph.D. granting

__ 4-year university or college, non-Ph.D. granting

__ Technical/community college

__ Other (please specify) _____________________________________________________________

4. What has your average ANNUAL teaching load been in the past two years?

___ 0 courses ___ 1–2 courses ___ >2–4 courses ___ >4–6 courses ___ >6 courses

5. How many teaching workshops have you ever attended, including the NETI?

___ 1–2 ___ 3–4 ___ 5–10 ___ >10

6. How many courses on teaching have you taken?

___ 0 ___ 1–2 ___ 3–4 ___ 5–10 ___ >10

7. On a 1 (poor) – 5 (excellent) scale, estimate your average student rating in undergraduate courses over

the past two years, rounding off to the nearest 0.1. Enter a zero if you have not taught any

undergraduate courses in the past two years.

______

8. On a 1 (poor) – 5 (excellent) scale, estimate your average student rating in graduate courses over the

past two years, rounding off to the nearest 0.1. Enter a zero if you have not taught any graduate

courses in the past two years.

______

9. What happened to your average student ratings in the years after you attended the NETI?

___ Decreased slightly ___ Increased slightly ___ Fluctuated or didn’t change much

___ Decreased moderately ___ Increased moderately ___ Not applicable

___ Decreased substantially ___ Increased slightly

10. What effect do you think the NETI had on your teaching ratings?

___ none ___ slight ___ moderate ___ substantial ___ my ratings didn’t change

11. What effect do you think the NETI had on your teaching (regardless of ratings)?

_________________________________________________________________________________

12. What is your involvement with providing instructional development (that is, giving teaching

workshops and/or seminars). Check the first response that applies.

___ Extensive in the past

___ Occasional in the past

___ None in the past, but plan to engage in it in the future

___ None in the past, and anticipate no future involvement

___ Other (please specify) ___________________________________________________________

13. Did the NETI motivate you to get involved in instructional development?

___ No ___ Yes

14. Rate your awareness and use of the concepts and teaching strategies listed below, checking the first

applicable button for each one.

Nev

er

heard

of

it

No

t s

ure

wh

at

it i

s

Ne

ve

r u

se

it

in

teach

ing

Use i

t o

ccasio

nally

Use i

t fr

eq

uen

tly

Learning styles O O O O O Learning objectives O O O O O Bloom’s Taxonomy O O O O O PowerPoint (or equivalent) O O O O O Active learning O O O O O Cooperative learning O O O O O Problem-based learning O O O O O Inquiry-based learning O O O O O Web-based instructional tutorials O O O O O Distance education O O O O O Robert Boice’s success strategies for new faculty O O O O O

15. What effect did the NETI have on your incorporation of the following concepts and strategies into

your teaching?

No

eff

ect

Slig

ht

eff

ect

Mo

dera

te e

ffect

Su

bsta

nti

al

eff

ect

Learning styles O O O O Learning objectives O O O O Bloom’s Taxonomy O O O O PowerPoint (or equivalent) O O O O Active learning O O O O Cooperative learning O O O O Problem-based learning O O O O Inquiry-based learning O O O O Web-based instructional tutorials O O O O Distance education O O O O Robert Boice’s success strategies for new faculty O O O O

16. How much have you been involved with research on teaching and learning? (Check all responses that

apply.)

___ I have done funded research and published results

___ I have done unfunded research and published results

___ I have done research and presented results at a conference

___ I have done classroom research to improve my teaching but never published or presented

___ I have never done educational research

17. Did attending the NETI motivate you to do research on teaching and learning?

___ No ___ Yes

18. Respond to the following questions

___ No ___ Yes Do you belong to the ASEE?

___ No ___ Yes Did the NETI motivate you to join the ASEE?

___ No ___ Yes Have you ever participated in an education conference?

___ No ___ Yes Did the NETI motivate you to participate in such conferences?

___ No ___ Yes Do you ever read education-related journal articles?

___ No ___ Yes Did the NETI motivate you to read more such articles?

19. Rate your agreement with the following statements.

Str

on

gly

dis

ag

ree

Dis

ag

ree

Neu

tral

Ag

ree

Str

on

gly

ag

ree

The most important characteristic of good teaching

Is to have complete and accurate lecture notes. O O O O O

Teachers should give students frequent course-

related group activities in class. O O O O O

Teachers should give students detailed study

guides for tests. O O O O O

Giving tests that only the top students have time to

complete sorts students effectively according to their

understanding of the material. O O O O O

Teachers should deliver lectures primarily via

transparencies or PowerPoint. O O O O O

It is acceptable for teachers to give assignments on

material that has not been explicitly covered in class. O O O O O

You can teach people to be critical thinkers. O O O O O

You can teach people to be creative thinkers. O O O O O

The next two questions are optional, but we’d be grateful if you respond to them.

20. Please share with us your recollections of and comments about the NETI and your evaluation of its

effect on your teaching.

________________________________________________________________________________

21. Do you have any suggestions for making the NETI more effective?

________________________________________________________________________________

We are requesting the following information for purposes of correlation. Your responses will be

integrated into a large statistical database, and no one but the survey coordinator will ever know how

you responded to any questions.

22. What is your name? (Last, First) This question is optional.

23. In what year did you attend the NETI? (Estimate if you’re not sure.) ________________

24. How many years had you taught before attending the NETI?

___ <1 ___ 1–2 ___ >2–5 ___ >5–10 ___ >10

25. What is your current position?

___ Full professor

___ Associate professor

___ Assistant professor

___ Lecturer/instructor

___ Graduate student

___ Other (please specify) _____________________________________________________

26. Are you

___ female ___ male

27. What are your principal responsibilities? (Check all that apply.)

___ Research

___ Undergraduate teaching

___ Graduate teaching

___ Administration

___ Advising/counseling

___ Instructional development

___ Other (please specify) ______________________________________________________

28. What is your principal academic discipline (in which you have done most of your teaching)?

__ Aerospace engineering

__ Bioengineering

__ Chemical engineering

__ Chemical engineering technology

__ Civil engineering (non-environmental)

__ Civil engineering (environmental)/Environmental engineering

__ Computer engineering

__ Computer science

__ Electrical engineering

__ Electrical engineering technology

__ Freshman engineering/general engineering

__ Industrial engineering/management science/operations research

__ Materials science/engineering

__ Mechanical engineering

__ Mechanical engineering technology

__ Nuclear engineering

__ Biological sciences

__ Physical sciences (chemistry, physics, earth & marine sciences)

__ Mathematical sciences (mathematics, statistics)

__ Other (please specify) ____________________________________________________

Thank you for taking the time to complete our survey.