April 2009 Journal of Engineering Education 145

Assessing General Creativity and CreativeEngineering Design in First YearEngineering Students

CHRISTINE CHARYTON

Department of PsychologyOhio State University

JOHN A. MERRILL

College of EngineeringOhio State University

ABSTRACT

Creativity is a vital tool for innovation in engineering. Psychologyand engineering faculty developed the Creative EngineeringDesign Assessment (CEDA) because existing tools are limited.This measure was administered with general creativity measuresin 63 engineering (57 males, six females) and 21 non-engineering(six males, 15 females) students in five week intervals. Inter-raterreliability showed high consistency overall and between the testand retest administrations. Only engineering males and femalessignificantly differed on the retest. Engineering students withlow, medium, and high creative engineering design did not statis-tically differ in their general creativity, not domain specific toengineering; however, only high scorers were significantly higheron the retest from the other groups. Future research is neededwith larger samples.

Keywords: creativity, engineering design, innovation

I. INTRODUCTION

Creativity research in engineering began to blossom in the

1950s (Ferguson, 1992). The recommendations of Vannevar Bush,

an electrical engineer from MIT, led to the establishment of the

National Science Foundation in 1950. In the early 1960s, the

National Science Foundation sponsored conferences on “scientific

creativity.” Yet, “as interest in engineering design faded in most

engineering schools, creativity was put on a back burner” (Ferguson,

1992, p. 57). Many engineering education programs appreciate

and value creativity, but few offer courses that teach about creativi-

ty. Creativity is defined as a preference for thinking in novel ways

and the ability to produce work that is novel and appropriate

(Sternberg, 1999; Weisberg, 1986) (see also Charyton, 2005). This

type of creativity can also be defined as “general creativity” that is

not domain specific (Charyton, 2005; Charyton and Snelbecker,

2007). Torrance (1974) stated that creativity has been formulated

in terms of a product (invention and discovery), process, a type of

person, and the production of something new to an individual or

culture. Creativity has been explored in relation to process, product,

personality, and press or environment (Sternberg, 1999; Sternberg

and Dess, 2001). Central themes specific to engineering creativity

include novelty and usefulness (Larson, Thomas, and Leviness,

1999; Nickerson, 1999; Thompson and Lordan, 1999). Engineer-

ing is an applied science where diagnostic procedures and problem-

solutions are derived (Schon, 1983). Innovation is a process to

place new ideas into practice where creativity acts as a vital tool

(Thompson and Lordan, 1999).

This study builds upon the current creativity research in psy-

chology and engineering education. The research project goals are

to provide a useful tool that can effectively assess creative engineer-

ing design at the university level in engineering education.

A. Practicality of CreativityMore recently, creativity has received greater attention as a

necessity, rather than an accessory in engineering design. “Creativity

is important to society, but it traditionally has been one of psychol-

ogy’s orphans” (Sternberg, 1999, p. 4). Even within psychology,

creativity has often been neglected. Czikszentmihalyi (1999) sug-

gested that the person, domain, and field are relevant to under-

standing creativity and innovation. Problem posing has been em-

phasized by the minds of many disciplines in art and science

(Smilansky and Halberstadt, 1986). Highly creative people rede-

fine problems, analyze ideas, persuade others and make reasonable

risks to help generate ideas (Sternberg and Dess, 2001). “Creativity

is certainly among the most important and pervasive of all human

activities. Homes and offices are filled with furniture, appliances,

and other conveniences that are products of human inventiveness”

(Simonton, 2000, p. 151). Engineering is a creative profession that

may be misunderstood. The ability to measure creativity would not

only facilitate the identification of talented individuals, but also

would allow the measurement of baseline information necessary to

track the progress of educational and training programs aimed at

enhancing creativity (Treffinger, 2003). “The need for people

skilled in helping others use creative problem solving is increasing”

(Isaksen, 1983, p. 18). This need is evident in both engineering and

the practice of psychology.

B. Engineering Creativity“Creativity, problem solving, and innovation are of increasing

concern to organizations in these times of accelerating change”

(Basadur and Finkbeiner, 1985, p. 37). The need for creativity,

problem solving, and innovation is becoming a global need. A

growing interest in the need and utilization of creativity in engi-

neering design is evident. Shirley interviewed by Elliott stated,

“I think if engineers are not creative, they’re not engineers” (Elliott,

2001, p. 22). How can an engineer be effective without creativity?

Designers have been engaged for centuries in engineering design;

however, only in the past few decades has there been a systematic

process capable of comprehensive analysis and improvement

(Soibelman and Pena-Mora, 2000). The need to assess and en-

hance creativity in engineering design is evident in many university

programs. There is a crucial need to teach about “real-world” engi-

neering design and operations that call for critical judgment and

creativity (Felder et al., 2000, p. 26). Creativity education is critical

in engineering education as well as general education (Ishii and

Miwa, 2005). Psychology is helpful to address creativity in educa-

tion by promoting learning through meta-cognition and self reflec-

tive activities (Ishii and Miwa, 2005). Students can gain confidence

to exercise reflection-in-action with a supportive learning environ-

ment that does not hinder their creativity (Green and Kennedy,

2001). Stimulating activities can encourage creativity and innova-

tion. Through incorporating knowledge from psychology into

engineering education, students can experience creative activities,

reflection, and their own awareness of their cognitive processes

(Ishii et al., 2006).

In the past, educational psychologists have assisted engineering

education faculty with enhancing learning for engineering students

(Felder, 1998). Empirical studies in educational and cognitive psy-

chology literature address methods for learning. These methods

have been implemented successively in engineering classes. Real-

world applications, cooperative learning, active learning, deductive,

and inductive learning are important for developing creativity.

Reflection-in-action is learning by doing (Schon, 1983). Students

also need to practice skills before they are assessed. Furthermore,

experiential learning provides students with opportunities to select

assignments and promotes deeper learning.

The goals and objectives in engineering education need to be

defined, clear and measurable (Felder and Brent, 2003). Creative

engineers are needed to solve technological problems. “It would

seem to be our responsibility to produce some creative engineers—

or at least not to extinguish the creative spark in our students”

(Felder, 1987, p. 222). Is it possible that engineering education de-

creases creativity and originality? To develop and nurture critical

and creative problem solving skills, we must provide opportunities

for students to exercise these skills. Open-ended questions, problem

finding, fluency (quantity of solutions), flexibility (variety of solu-

tions), and originality (novelty) are vital components toward

enhancing analysis and synthesis of information learned (Felder,

1987; Isaksen and Parnes, 1985). Shah, Smith, and Vargas-

Hernandez (2003) also describe these three constructs (fluency,

flexibility, and originality) under different terms such as quantity

(total number of ideas generated) variety (areas of the solution

space), and quality (feasibility of an idea to meet the design specifi-

cations). According to Shah et al., novelty is an approach to mea-

sure effectiveness that relates to quality.

Central themes specific to engineering creativity include origi-

nality (novelty) (Shah, Smith, and Vargas-Hernandez, 2003;

Thompson and Lordan, 1999; Weisberg, 1999) and usefulness

(applicability) (Larson, Thomas, and Leviness, 1999; Shah, Smith,

and Vargas-Hernandez, 2003; Thompson and Lordan, 1999).

Engineers not only need to address aesthetics like artists, but also

need to solve problems, prevent potential problems, and address

utility within the constraints and parameters that are designated.

Furthermore, creativity as an aspect of engineering can be referred

to as “functional creativity” (Cropley and Cropley, 2005). Functional

creativity means that products designed by engineers typically serve

a functional and useful purpose, unlike fine art. Creative products

emphasize novelty, resolution, elaboration, and synthesis (Cropley

and Cropley, 2005). Furthermore, problem finding may offer

another avenue to increase creative production (Nickerson, 1999).

Problem finding is a skill often found in art, yet is also necessary in

science and engineering. Both problem finding and problem solv-

ing are relevant to an engineer’s creativity; however, these attributes

have not been measured in great depth in engineering creativity

specifically. Such attributes need to be assessed and further devel-

oped by appropriate educational intervention activities (Cropley

and Cropley, 2005). The need to measure these attributes in indi-

viduals and teams would be appropriate and beneficial.

Measuring creativity in engineering design is necessary to assess

how these skills are demonstrated and developed in engineering

programs. Engineering students may profit by understanding con-

straints through reflective learning that are necessary to be creative

in the engineering field. Students may also become aware of their

own meta-cognitive processes to enhance their skills in engineering

design (Ishii and Miwa, 2005; Ishii et al., 2006).

Charyton (2005) and Charyton and Snelbecker (2007) investi-

gated measures to assess creativity in engineering students. Like the

work of Basadur and Finkbeiner (1985), Charyton and Snelbecker

(2007) explored much of the literature in creativity assessment and

consulted psychologists and engineers assessing creativity. Conclu-

sions indicated that measures such as the Myers Briggs Type Indi-

cator (MBTI) did not specifically address creativity in engineering

or engineering design. The MBTI has limitations and does not

specifically assess creativity (Larson, Thomas, and Leviness, 1999).

To date, existing engineering creativity measures are limited.

According to the literature available, there are few measures to as-

sess creative abilities in engineering design. The Owens Creativity

Test (Owens, 1960) was developed to assess mechanical engineer-

ing design. Test takers list possible solutions to mechanical prob-

lems. Its reliability ranged from 0.38 to 0.91 and its validity ranged

from 0.60 to 0.72, when applied to engineers in mechanically

related occupations. This assessment tool is out of print and is no

longer used.

Lawshe and Harris (1960) developed the Purdue Creativity Test

as an engineering personnel test, for identifying creative engineers

and their occupational potential. Participants are instructed to list as

many possible uses for one or two shapes that are provided. This as-

sessment has adequate reliability (0.86 to 0.95) and modest validity

(29 percent to 73 percent for low scorers and high scorers, respec-

tively). Validity was determined by assessing professional engineers

(product, process, and product engineers) working in industry.

Participants are instructed to generate original and novel possible

uses for single objects or pairs of objects. Scoring is based on fluency

(number of uses), and flexibility (differing categories of uses). Their

test does not directly assess originality. Although a reliable and valid

measure, limitations include little use in the field of engineering.

This assessment measures engineering creativity by assessing fluency

(number of responses) and flexibility (categories of responses). Both

the Owens Test and the Purdue Creativity Test only measure

divergent thinking.

Traditional divergent tests (e.g., the Purdue Creativity Test, and

the Owens Creativity Test) only measure lists of possible uses. The

146 Journal of Engineering Education April 2009

April 2009 Journal of Engineering Education 147

Creative Engineering Design Assessment (CEDA) offers a new

method for assessing creative engineering design. Participants are

asked to sketch designs that incorporate one or several three-

dimensional objects, list potential users (people), and perform prob-

lem finding (generate alternative uses for their design) as well as

problem solving in response to specific functional goals. Sketching

is instrumental in design problem solving (Goldschmidt and

Smolkov, 2006) and results in creative solutions. Sketching is useful

for more creative results due to experience for spatial manipulations

that are domain specific. Design is crucial for creativity and innova-

tion for users and customers (Cockton, 2008). Engineering creativity

involves both convergent and divergent thinking. The CEDA

measures both convergent (generating a solution to the problem

posed) and divergent thinking (generating multiple solutions to

problems posed). Constraint satisfaction is assessed by measuring

the amount of shapes used as well as the materials added to each de-

sign. Schon (1983) also reported that Schein segregates convergent

science from divergent practice. Furthermore, Schein relegated

divergence to a residual category as a skill that is present in minor

professions, compared to convergence in major professions. He

stated that major professions include medicine and engineering

while other professions, such as education and social science are

attracted by the major professions as models. The study of creativity

in psychology has traditionally emphasized divergent thinking skills

(Torrance, 1974; Guilford, 1984). In the CEDA model, conver-

gent science and divergent practices are integrated as necessary

functions of cognitive processes that are assessed for creative

engineering design.

The CEDA measures constructs that are a part of the design

process as a step by step process, beginning with sketching a design

for each problem. Figure 1 shows the theoretical rationale of con-

structs necessary for the creative design process and for the selection

of instruments to assess creativity in engineering design. This figure

is based on previous studies to assess creativity as defined by the

person, process, product, and environment (Clapham, 2001;

Sternberg, 1999) and conceptualization of necessary creative

processes in engineering design (Cropley and Cropley, 2005; Finke,

Ward, and Smith, 1992; Stokes, 2006) that are domain specific

(Kaufman and Baer, 2005; Nickerson, 1999). Problem finding,

problem solving, divergent thinking, convergent thinking, and

constraint satisfaction are necessary in the creative process of engi-

neering design (see also Charyton, Jagacinski, and Merrill, 2008).

Our model, depicted in Figure 1 illustrates that personal attributes

of the individual defined as personality, temperament, and cognitive

risk tolerance influence one’s creative process. The environment of

the individual, defined as the engineering classroom or industrial

setting, also influences the creative process. The creative process is

defined as using divergent thinking, convergent thinking, con-

straint satisfaction, problem solving, and problem finding to create

a design. The creative process directly affects the product design. At

the same time, the product design dialectically influences the cre-

ative process. The product is shaped through the creative process.

At the same time, product development influences the creative

process. Last, the CEDA is a measure of the product design that is

developed by the creative process. Each portion of each problem

directly relates to psychological constructs of engineering creativity

that are described from the literature.

Figure 2 describes the theoretical rationale for the test construc-

tion of the CEDA, based on creativity literature specific to engi-

neering creativity shown in Figure 1. Figure 2 shows how each item

on the CEDA addresses these theoretical constructs. Divergent

thinking is assessed by generating multiple solutions. Convergent

Figure 1. Conceptualization of measures addressing creative mechanisms in engineering design (Charyton, Jagacinski, andMerrill, 2008).

148 Journal of Engineering Education April 2009

thinking is assessed by solving the problem posed. Constraint satis-

faction is assessed by complying with the parameters of the direc-

tions and also adding additional materials and manipulating the

objects as desired. Problem finding is assessed by identifying other

uses for the design. Problem solving is assessed by deriving a novel

design to solve the problem posed.

The readability and comprehension of the CEDA is appropriate

for college students. The Simple Measure of Gobbledygook

(SMOG) (McLaughlin, 1969) online program was used to assess the

reading and comprehension level of the CEDA, using established

readability formulas, available at: http://www.harrymclaughlin.com/

SMOG.htm. This formula provides the widest range of educational

level and ability to match scores to actual education level. The

online SMOG calculator uses McLaughlin’s formula yielding a

0.985 correlation with the grade level of readers having 100 percent

comprehension of the tested materials. The SMOG is designed for

evaluating the reading level of materials that can be read indepen-

dently by a person without assistance from a teacher or instructor

(Richardson and Morgan, 1990). Readability is recommended at

the sixth to seventh grade level for educational materials for the

general public, being equivalent to junior high school. The SMOG

Grade for the CEDA was 8.81, being the 8th grade level, equiva-

lent to a junior high school reading and comprehension level.

Therefore the CEDA would also be appropriate and useful for

pre-college students at the high school level.

Our goals were to assess creativity through the newly developed

Creative Engineering Design Assessment (CEDA) in male and

female engineering students for comparison with male and female

non-engineering students. Our objectives were to establish test-

retest reliability of this assessment tool. Our research questions

included: (1) what is the relationship between the CEDA and other

general creativity measures in terms of a) test-retest consistency and

b) what is the relationship between creative engineering design

compare and general creativity; (2) what is the inter-rater agreement

between judges when using the new assessment tool on fluency,

flexibility, originality, and the overall CEDA; (3) what are the simi-

larities and differences between male and female engineering

students and male and female non-engineering students in terms of

both general creativity and creative engineering design; and (4) how

do engineering students with higher creative engineering design

compare with others on general creativity and creative engineering

creativity within a five week interval?

II. METHODOLOGY

Sixty one first year engineering students (49 percent were

freshmen or sophomores) consisting of 57 males and six females;

and 21 one non-engineering students (95 percent of students were

freshmen or sophomores) consisting of six males, and 15 females at

a mid-western U.S. university were administered the newly devel-

oped Creative Engineering Design Assessment (CEDA) and a

demographic questionnaire with established general creativity (not

specific to engineering creativity) measures including: (1) the Creative

Personality Scale (CPS), (2) the Creativity Temperament Scale

(CT), and (3) the Cognitive Risk Tolerance Scale (CRT) during a

fundamentals of engineering five week summer course. The engi-

neering course consisted of basic skills and a laboratory design

course to engage students in a quarter long design project. This

course is a part of the First Year Engineering Education curriculum.

The course contained hands on design activities, where the final

project objective was to design a functional roller coaster. The

course also consisted of sketching exercises and journal assign-

ments. Learning outcomes were to gain sketching and design skills

necessary to engineering.

Non-engineering students were from various majors and were

recruited as volunteers from an introductory psychology course

where they were encouraged to participate in research studies. Non-

engineering students did not have the same roller coaster project,

the same sketching exercises, or the same journal assignments.

Learning objectives and outcomes of the psychology course were to

gain knowledge about the various areas and fields in psychology.

Students were not matched based on performance, grade point

average, Scholastic Achievement Test scores, or other performance

variables. Both groups were assessed in a five week interval. Partici-

pants were given a number for identification purposes according to

the Institutional Review Board approved procedures.

III. INSTRUMENTS

A questionnaire was administered requesting demographic

information. This included age, gender, ethnicity, major, and spe-

cialization area within the major. Additional questions included

“How often does an engineer use creativity in a given day?” and

“How often do you use creativity in a given day?”

Figure 2. Creative engineering design assessment meta-cognitive processes measured.

April 2009 Journal of Engineering Education 149

1) CPS: Creative Personality Scale: The Creativity Personality

Scale (CPS) of the Adjective Checklist (ACL) (Gough, 1979) was

administered to assess creativity attributes. This test for creative

thinking was chosen because it is highly regarded, reliable, and

widely used as a creativity test. For example, Plucker and Renzulli

(1999, p. 46) stated that, “ Oldham and Cummings (1996, p. 609),

in a comparison of personality traits, environmental characteristics,

and product ratings, found evidence that people with specific per-

sonality traits (i.e., as judged by Gough’s (1979) Creativity Person-

ality Scale) produced creative products when challenged by their

work and supervised in a supportive ‘non-controlling fashion’.”

Furthermore, normative data for the mean scores of 66 engineering

students was 3.88 with a standard deviation of 3.94 compared to

256 males 3.57 with a standard deviation of 3.99 (Gough, 1979).

2) CT: Creative Temperament Scale: The Creative Temperament

Scale (Gough, 2000) adapted from the California Psychological

Inventory (CPI) was designed to assess personality characteristics

and predict what people will say and do in specific contexts. The

Creative Temperament Scale is one of the special purpose scales of

the CPI. Gough, gave permission to extract the CT scale from the

CPI (H. Gough, personal communication, June 28, 2003). Norma-

tive data for the mean scores of 66 engineering students was 24.55

with a standard deviation of 4.98 compared to 22.65 and 5.46,

respectively, for the general normative data (n � 3,235) (Gough,

2000).

3) CRT: Cognitive Risk Tolerance Survey: The Cognitive Risk

Tolerance Survey (Snelbecker, McConologue, and Feldman) “con-

sists of 35 self report items designed to assess an individual’s ability

to formulate and express one’s ideas despite the threat of negative

assessment regarding: reputation, integrity, credibility, honor and

intelligence” ( J. Feldman [formerly J. Teitlebaum], personal com-

munication, May 27, 2003). Responses are on a Likert Scale rang-

ing from 0 (Very Strongly Disagree) to 9 (Very Strongly Agree).

Higher scores indicate higher levels of cognitive risk tolerance. The

Cognitive Risk Tolerance Survey was developed as an extension of

an earlier risk tolerance model developed by Snelbecker and col-

leagues (Roszkowski, Snelbecker, and Leimberg, 1989; Snelbecker,

Roszkowski, and Cutler, 1990). Reliability of this psychometric

instrument has proven to be adequate with a Cronbach’s alpha coef-

ficient of 0.76 during Feldman’s (2004) pilot study with 78 respon-

dents, and a Cronbach’s alpha coefficient of 0.78 during the main

section of her dissertation study with 84 respondents.

4) Creativity Engineering Design Creativity (CEDA): The new

assessment tool—the Creativity Engineering Design Assessment

(see also Charyton, Jagacinski, and Merrill, 2008) consists of five

design problems with five parts each to assess an individual’s ability

to formulate and express design ideas through sketching, providing

descriptions, identifying materials, and identifying problems that

the design solves and its potential users. Instructions are to generate

as many designs with at least one design per problem. Additionally,

at least one response should be indicated for each of the five ques-

tions for each design. Total time for this assessment is 25 minutes

for five pages, or about five minutes per page. Appendix A contains

a sample problem (see also Charyton, Jagacinski, and Merrill,

2008).

Dimensions of the assessment tool included both problem solving

and problem finding exercises. Participants were evaluated according

to their originality (0–10), fluency (amount of ideas), and flexibility

(differing types of ideas) for each design problem. Appendix B

contains scoring criteria (see also Charyton, Jagacinski, and Merrill,

2008).

Two judges were selected: one from engineering and one from

psychology. Two of the CEDA test developers trained the judges.

Judges practiced scoring in a team environment; however, each

judge evaluated the CEDAs separately. Judges evaluated: fluency

(amount of items listed), flexibility (categories of responses per

problem), and originality (on an eleven point scale found in

Appendix B). Student identification numbers were hidden from

the judges.

IV. RESULTS

1) The CEDA in comparison with other general creativity measures:(a) Correlations between the test and retest in the five week

interval were conducted to establish the reliability of the

CEDA in comparison with other established creativity

measures. The CEDA was consistent for the test and retest

(r � 0.563) like the other general creativity measures such

as the CPS (r � 0.569), CT (r � 0.512) and CRT

(r � 0.428), p � 0.01 for all comparisons.

(b) Correlations among the instruments were conducted to

identify their relationships with the CEDA. The CEDA

has low correlations with the CPS (r � �0.007), CT

(r � �0.131), and the CRT (r � �0.187) for both the test

and the retest CPS (r � �0.119), CT (r � �0.032), and

CRT (r � �0.159), p � 0.05 for all comparisons.

2) Interrater reliability: Correlations among the CEDA scores of

two judges were conducted to identify their relationships with each

other and establish reliability. The judges were in agreement

(r � 0.98) with their overall test and retest scoring. Inter-rater relia-

bility for flexibility (r � 0.90, r � 0.98) and originality (r � 0.80;

r � 0.85) indicated consistency in both test and retest measures,

respectively. Therefore, both judges’ scores were average for com-

parison and analysis.

3) Similarities and differences in creativity assessments: Similarities

and differences in general creativity in engineering students and

non-engineering students were examined. A MANOVA was used

to detect differences in three general creativity measures (creative

personality, creative temperament, and cognitive risk tolerance).

Figure 3 contains a depiction of the means for male and female

engineering, and male and female non-engineering students on the

Creative Engineering Design Assessment (CEDA) compared with

the following general creativity variables: (1) Creativity Personality

Scale (CPS), (2) Creativity Temperament Scale (CT), (3) Cognitive

Risk Tolerance Scale (CRT). No significant differences were found

between engineering students and non-engineering students.

Figure 4 contains a depiction of the mean retest scores for low

(1–84.5), medium (85–136.5), and high (137–300) creative engi-

neering design (based on the CEDA test scores) for engineering

students on the CEDA retest compared with the following general

creativity variables: (1) Creativity Personality Scale (CPS), (2)

Creativity Temperament Scale (CT), (3) Cognitive Risk Tolerance

Scale (CRT). No significant differences were found for engineering

students.

a) General creativity: A two-way MANOVA was calculated

examining the effect of class and gender on general creativity

measures before and after their class. No significant differences

150 Journal of Engineering Education April 2009

were found in terms of general creativity for engineering and

non-engineering students (F(6) � 0.718, p � 0.636), gender

(F(6) � 0.633, p � 0.704) or their interaction (F(6) � 0.831,

p � 0.550).

b) Creative Engineering Design: Kruskal Wallis analyses were

calculated examining the effect of class and gender on creative engi-

neering design. A significant difference was found for creative

engineering design on the CEDA test �2 � 8.392, p � 0.039, and

CEDA retest �2 � 24.70, p � 0.000 for engineering students.

Follow-up analyses were completed through Mann-Whitney

analyses. A significant difference was found only for male and

female engineering students for the retest only, p = 0.008 indicating

that male engineering students tended to perform at lower levels

in comparison with the female engineering students who had

significantly higher creative engineering design in the CEDA

retest only. No significant differences were found between male and

female engineering students and male and female non-engineering

students.

4) High creative engineering design compared with low and mediumcreative design based on CEDA test scores: Based on creative engineer-

ing design for high creative engineering design engineering

students, low, medium, and high scorers were compared on the

retest. No significant differences were found for general creativity

(F(6) � 1.30, p � 0.261). Since ANOVA results did not indicate

homogeneity, Welch and Brown-Forsythe analyses were performed

to compare creative engineering design. A significant difference was

Figure 3. Descriptive statistics of means for creative engineering design and general creativity measures by group and gender.�indicates statistically significant differences

Figure 4. Descriptive statistics of means for engineering students for creative engineering design and general creativity measures bylow, medium, and high overall CEDA test scores.�indicates statistically significant differences

found Welch (F(2) � 5.56, p = 0.009); Brown-Forsythe (F(2) �6.22, p � 0.005). Tukey posthoc analyses revealed that high creative

engineering design scorers significantly demonstrated higher cre-

ative engineering design than low scorers (p � 0.001) and medium

scorers (p � 0.038). Paired samples t-tests were performed to com-

pare test and retest differences of all measures. Differences were not

detected on the CPS (p � �0.206, p � 0.838), CT (t � �1.166,

p � 0.255), CRT (t � �0.884, p � 0.385), or CEDA (t � 0.361,

p � 0.721) for low CEDA scorers. Differences were not detected

on the CPS (t � �1.349, p � 0.192), CT (t � 0.452, p � 0.656),

or CRT (t � �0.377, p � 0.710) for medium or high scorers on

the CPS (t � 0.431, p � 0.673), CT (t � 2.082, p � 0.056), or

CRT (t � �1.102, p � 0.289). However, significant differences

were found on the CEDA test and retest for medium CEDA

scorers (t � 3.653, p � 0.001), and high CEDA scorers (t � 3.443,

p � 0.004).

V. DISCUSSION

Our first research question concerned reliability of the CEDA

in comparison with established general creativity measures. The

CEDA demonstrated statistically significant consistency, like the

established general creativity measures between test and retest

scores (Charyton, Jagacinski, and Merrill, 2008). Furthermore, the

CEDA was not related to the general creativity measures. This

finding supports the notion that creative engineering design is dif-

ferent than general creativity. Other studies (Charyton and Snel-

becker, 2007) found that music improvisation (music creativity)

was found to be unrelated to general creativity. The Purdue Cre-

ativity Test, a test of engineering creativity, was found to be mod-

estly related to general creativity measures, suggesting that the

Purdue Creativity Test may have been evaluating similar con-

structs to general creativity. Furthermore, other divergent thinking

tests, like the Purdue Creativity Test, the Owens Creativity Test,

the Torrance Test for Creative Thinking, and the Structure of In-

tellect Model measure divergent thinking, which is a component

of creativity. The CEDA measures divergent thinking, but also

measures convergent thinking—both are necessary for creativity in

engineering design. This study demonstrated how creative engi-

neering design is different from general creativity constructs; how-

ever, more research is needed to assess how creative engineering

design is similar and or different than other divergent thinking

tests in engineering.

Agreement among judges was high for fluency, flexibility, origi-

nality, and overall scores. Fluency reliability was high, due to the na-

ture of the construct being counting responses. Flexibility was still

high relating to categories of responses. Last, originality was high

measuring the quality of originality that the designs produced

solved the given problem. Some responses were more typical,

indicating a lower rating, while other designs demonstrated higher

novelty and unusual responses, indicating a higher rating. We

speculate that high reliability for originality may be due to the com-

bination of descriptors and numeric values in the scoring rubric.

Judges were trained to assess the designs and generate their own

reaction to the designs, then select the word and appropriate num-

ber to describe their analysis. It is plausible that the combination of

verbal and numeric ratings is a potential reason for the high inter-

rater agreement. The high agreement of judges was encouraging

since both the engineer and psychologist were in high agreement

for originality (novelty) as well as fluency, flexibility, and overall

scores. The reliability for the CEDA was similar to previous stud-

ies, indicating high inter-rater agreement (Charyton, Jagacinski,

and Merrill, 2008).

Similarities and differences were found among engineering stu-

dents in terms of general creativity and creative engineering design.

First, engineering students did not differ in their overall levels of

general creativity compared with non-engineering students. General

creativity was measured by creative personality, creative tempera-

ment, and cognitive risk tolerance in five week test and retest inter-

vals. Although we anticipated that both male and female engineers

may have higher levels of creative engineering design on the retest

due to the nature of the engineering course and factors related to

retesting, male engineering students had significantly lower creative

engineering design only during the retest in comparison with only

the female engineering students. There were no significant differ-

ences between the male and female engineering and male and female

non-engineering students. We do not know why only engineering

males decreased; however, lower scores for male engineering

students was also found in a previous study (Charyton, Jagacinski,

and Merrill, 2008). This trend will be examined in further research.

We speculate that a decreased score may be due to students learning

new design techniques, since only the engineering students were

exposed to sketching and a rollercoaster project in their design

course. However, this finding did not affect the female engineering

students in the same manner. We plan to investigate this phe-

nomenon in a larger sample size to determine if there are gender

differences in engineering students or if acquiring new design skills

interacts with creative engineering skills. When some students learn

a new skill, there may be a refractory period when reproducing that

skill, since they are applying a new method (Bransford, Brown, and

Cocking, 2000; Kidder 1981). This may explain a decrease in their

retest CEDA scores. However, medium and high engineering stu-

dent scorers on the CEDA also tended to have significantly lower

scores in the second retest administration. Therefore, it may be likely

that the combination of time constraints and re-exposure to generate

novel and original designs to the same problems were difficult for

some students such as the engineering male students, medium, and

high scorers. For future studies assessing creative engineering design,

longer time durations between administrations with less time

constraints may be more helpful to engineering students.

Limitations of this study include a small sample size. In our

study, we had fewer female engineering students compared to

male engineering students. However, females account for less than

20 percent of overall engineering students. Future research will

investigate similarities and differences among engineering students

in larger sample sizes. With a larger sample size, we may be able to

investigate similarities and differences among engineering students

by gender in greater detail. This study did not compare the CEDA

with other engineering creativity measures, such as the Purdue

Creativity Test. Further research is needed to compare the CEDA

with the Purdue Creativity Test. Future studies will assess creativity

outcomes in longer time durations.

High creative engineering design students statistically differed

from low and medium scorers in creative engineering design,

despite scores decreasing for medium and high scorers in the retest

administration. Various authors have contended that creativity has

been neglected or dismissed as being less important in curriculum.

April 2009 Journal of Engineering Education 151

152 Journal of Engineering Education April 2009

This is of particular interest since we speculated that if students

are given the exposure, such as hands on design training, they may

develop and hone creativity and innovation skills. Furthermore, it

is plausible that once creative skills are learned, students may con-

tinue to demonstrate these skills in comparison with those who do

not receive hands on activities in engineering design. Further-

more, this assessment tool may offer an option to assess creativity

in engineering university programs (Soibelman and Pena-Mora,

2000). Through teaching and measuring creativity in the engi-

neering curriculum, students may have the opportunity to hone

creative engineering design skills by practicing. Like some creativ-

ity researchers (Torrance 1974), the test developers of the CEDA

believe that creativity is a skill that can be developed through a

supportive learning environment. This viewpoint is consistent

with other available literature in engineering education, indicating

that if students are expected to demonstrate skills, they need the

opportunity to practice (Felder, 1988; 1987; Felder et al., 2000;

Felder and Brent, 2003; Isaksen and Parnes, 1985). Thus, both

creativity researchers and engineering educators are in agreement

for the need to support and facilitate creativity in education

through a supportive environment with opportunities for reflective

practice (Ishii and Miwa, 2005; Schon, 1983). Furthermore, these

necessary skills for innovation could be assessed for instructor

feedback of student learning.

The newly developed assessment tool differed from the other

general creativity measures, like previous studies (Charyton,

Jagacinski, and Merrill, 2008) indicating differences in the CEDA

and similarities in general creativity. Attributes such as tempera-

ment and personality may tend to be more stable traits (Charyton

and Snelbecker, 2007; Gough, 1979; Oldham and Cummings,

1996). Charyton and Snelbecker (2007) proposed that cognitive

risk tolerance may be a component of general creativity that is likely

to be moderately related to, but different from, other general

creativity measures that may be more flexible, depending on an in-

dividual’s comfort level in various situations or conditions of uncer-

tainty. Furthermore, it is plausible that there may also be variation

in creative engineering design, since students can learn and develop

design skills in the engineering classroom. Students may learn

creativity through hands on activities that foster meta-cognition

processes about creativity and design (Ishii and Miwa, 2005).

CONCLUSIONS AND IMPLICATIONS

Students need the opportunity to practice creative design skills.

Faculty need to offer opportunities to develop this skill and assess

students on their progress. Perhaps once this skill has been devel-

oped, students can continue to demonstrate this skill. The CEDA

offers a tool of assessment to measure the challenging construct of

creativity in engineering design, which is a necessary skill for inno-

vation. To date, few tools exist to assess this construct. Since cre-

ativity is a necessary skill to be innovative and many engineering

programs strive to produce innovative engineers, this tool has a

practical application that could benefit engineering education.

Furthermore, skills that foster innovation, such as creativity, should

be a component of the curriculum so that students can practice and

develop these skills.

Continued research development on this instrument by engi-

neering and psychology faculty can benefit students within our

university and at other universities. Some researchers have indicated

the need to assess creativity in engineering classes (Cropley and

Cropley, 2005; Felder, 1987; Felder et al., 2000). The CEDA tool

could be utilized for educational purposes. Furthermore, creativity

as a component of the engineering curriculum could be an interven-

tion to provide students with more opportunities to develop these

necessary skills that are a part of being an engineer. Through under-

standing the value and nature of usefulness (Larson, Thomas, and

Leviness, 1999; Nickerson, 1999) as a key component of engineer-

ing creativity, we can enhance our understanding of creative

processes in engineering and in other domains. The understanding

of meta-cognition and the engineering creativity process will lead

engineering educators and creativity researchers to complement

each other toward increasing both the frequency and quality of

inventiveness. Creativity is key for innovation in industry. Inven-

tiveness can benefit many human conveniences (Simonton, 2000).

Thus, by providing a method for assessing creativity in engineering

design, educators can enable students to develop their talents as

future innovative engineers.

ACKNOWLEDGMENTS

The authors would like to thank Dan Wisniewski, who assist-

ed as a judge; Glenn Elliott for assisting with graphic design tech-

nology for our assessment tool; John Elliott for his critical reading

of the manuscript and critical assistance with data management;

Richard Jagacinski for his valuable feedback on the CEDA design

and Mohammed A. Rahman for statistical consultation. Prelimi-

nary data were presented as invited talks at the National Science

Foundation in Arlington, Virginia and the American Psychologi-

cal Association Convention in New Orleans, Louisiana. Prelimi-

nary data were also presented as a symposium on Creativity and

Innovation at the Midwestern Psychological Association conven-

tion in Chicago, Illinois.

REFERENCES

Basadur, M., and C.T. Finkbeiner. 1985. Measuring preference for

ideation in creative problem-solving training. Journal of Applied Behavioral

Science 21 (1): 37–49.

Bransford, J.D., A.L. Brown, and R.R. Cocking. 2000. How people

learn: Brain, mind, experience, and school. Washington, DC: National

Academies Press.

Charyton, C. 2005. Creativity (scientific, artistic, general) and risk tolerance

among engineering and music students. Ph.D. dissertation. Philadelphia, PA:

Temple University.

Charyton, C., R.J. Jagacinski, and J.A. Merrill. 2008. CEDA: A re-

search instrument for creative engineering design assessment. Psychology of

Aesthetics and Creativity in the Arts 2 (3): 147–54.

Charyton, C., and G.E. Snelbecker. 2007. General, artistic and scien-

tific creativity attributes of engineering and music students. Creativity

Research Journal 19 (2–3): 213–225.

Clapham, M.M. 2001. The effects of affect manipulation and informa-

tion exposure on divergent thinking. Creativity Research Journal 13 (3–4):

335–50.

Cockton, G. 2008. Designing worth- Connecting preferred means to

desired ends. Interactions 15 (4): 54–57.

Cropley, D., and A. Cropley. 2005. Engineering creativity: A sys-

tems concept of functional creativity. Mahwah, NJ: Lawrence Erlbaum

Associates.

Csikszentmihalyi, M. 1999. Implications of a systems perspective for

the study of creativity. In Handbook of Creativity, ed. R.J. Sternberg.

Cambridge, MA: MIT Press.

Elliott, M. 2001. The well-rounded IE: Breakthrough thinking. IE

Solutions. Oct: 22–25.

Felder, R.M. 1987. On creating creative engineers. Engineering Educa-

tion 77 (4): 222–27.

Felder, R.M., and L. Silverman. 1988. Learning and teaching styles in

engineering education. Engineering Education 78 (7): 674–81.

Felder, R.M., G. Felder, and J. Dietz. 1998. A longitudinal study of

engineering student performance and retention. V. Comparisons with

traditionally-taught students. Journal of Engineering Education 87 (4):

469–80.

Felder, R.M., and R. Brent. 2003. Designing and teaching courses to

satisfy the ABET engineering criteria. Journal of Engineering Education

92 (1): 7–25.

Felder, R.M., D.R. Woods, J.E. Stice, and A. Rugarcia. 2000. The

future of engineering education. II. Teaching methods that work. Chemical

Engineering Education 34 (1): 26–39.

Feldman, J.M. 2004. The relationship among college freshmen’s cognitive risk

tolerance, academic hardiness, and emotional intelligence and their usefulness in

predicting academic outcomes. Ph.D. dissertation. Philadelphia, PA: Temple

University.

Ferguson, E.S. 1992. Engineering and the mind’s eye. Cambridge, MA:

MIT Press.

Finke, R.A., T.B. Ward, and S.M. Smith. 1992. Creative cognition:

Theory, research, and applications. Cambridge, MA: MIT Press.

Goldschmidt, G., and M. Smolkov. 2006. Variances in the impact

of visual stimuli on design problem solving performance. Design Studies

27 (5): 549–69.

Gough, H.G. 2000. The California Psychological Inventory. Mahwah,

NJ: Lawrence Erlbaum Associates.

Gough, H.G. 1979. A creative personality scale for the Adjective

Check List. Journal of Personality and Social Psychology 37 (8): 1398–1405.

Green, G., and P. Kennedy. 2001. Redefining engineering education:

The reflective practice of product design engineering. International Journal

of Engineering Education 17 (1): 3–9.

Guilford, J.P. 1984. Varieties of divergent production. Journal of

Creative Behavior 18 (1): 1–10.

Isaksen, S.G. 1983. Toward a model for the facilitation of creative

problem solving. Journal of Creative Behavior 17 (1): 18–31.

Isaksen, S.G., and S.J. Parnes. 1985. Curriculum planning for cre-

ative thinking and problem solving. Journal of Creative Behavior 19 (1):

1–29.

Ishii, N., and K. Miwa. 2005. Supporting reflective practice in creativity

education. In Proceedings of the 5th conference on Creativity & Cognition.

London, England.

Ishii, N., Y. Suzuki, H. Fujiyoshi, T. Fujii, and M. Kozawa. 2006. A

framework for designing learning environments fostering creativity. In

Current developments in technology-assisted education, eds. A. Méndez-Vilas,

A. Solano Martín, J.A. Mesa González, and J. Mesa González, 228–232.

Badajoz, Spain: FORMATEX.

Kaufman, J.C., and J. Baer, eds. 2005. Creativity across domains: Faces of

the muse. Mahwah, NJ: Lawrence Erlbaum Associates.

Kidder, Tracy. 1981. The soul of a new machine. 1st ed. Boston, MA:

Little, Brown.

Larson, M.C., B.H. Thomas, and P.O. Leviness. 1999. Assessing

creativity in engineers. American Society of Mechanical Engineers, Design

Engineering Division (Publication) DE 102: 1–6.

Lawshe, C.H., and D.H. Harris. 1960. Manual of instructions to

accompany Purdue creativity test forms G and H. Princeton, NJ: Educational

Testing Services.

McLaughlin, G.H. 1969. SMOG grading: A new reading formula.

Journal of Reading 12 (8): 639–40.

Nickerson, R.S. 1999. Enhancing creativity. In Handbook of creativity,

ed. Robert J. Sternberg. Cambridge, MA: Cambridge University Press.

Oldham, G.R., and A. Cummings. 1996. Employee creativity: Personal

and contextual factors at work. Academy of Management Journal 39 (3):

607–34.

Owens, W.A. 1960. The Owens’ creativity test. Ames: Iowa State

University Press.

Plucker, J.A., and J.S. Renzulli. 1999. Psychometric approaches to the study

of human creativity. New York: Cambridge University Press.

Richardson, J.S., and R.F. Morgan. 1990. Reading to learn in the content

areas. Belmont, CA: Wadsworth Publishing Co.

Roszkowski, M.J., G.E. Snelbecker, and S.R. Leimberg. 1989. Risk

tolerance and risk aversion. In The tools and techniques of financial planning,

ed. Stephan R. Leimberg. Cincinnati, OH: The National Underwriter

Company.

Schon, D.A. 1983. The reflective practitioner: How professionals think in

action. New York: Basic Books.

Shah, J.J., S.M. Smith, and N. Vargas-Hernandez. 2003. Metrics for

measuring ideation effectiveness. Design Studies 24 (2): 111–34.

Simonton, D.K. 2000. Creativity: Cognitive, personal, developmental,

and social aspects. American Psychologist 55 (1): 151–58.

Smilansky, J., and N. Halberstadt. 1986. Inventors versus problem

solvers an empirical investigation. The Journal of Creative Behavior 20 (3):

183–201.

Snelbecker, G.E., T. McConologue, and J.M. Feldman. Cognitive risk

tolerance survey. Unpublished manuscript.

Snelbecker, G.E., M.J. Roszkowski, and N.E. Cutler. 1990. Investors’

risk tolerance and return aspirations, and financial advisors’ interpretations:

A conceptual model and exploratory data. Journal of Behavioral Economics

19 (4): 377–93.

Soibelman, L., and F. Pena-Mora. 2000. Distributed multi-reasoning

mechanism to support conceptual structural design. Journal of Structural

Engineering 126 (6): 733.

Sternberg, R.J. 1999. Handbook of creativity. New York: Cambridge

University Press.

Sternberg, R.J., and N.K. Dess. 2001. Creativity for the new millennium.

American Psychologist 56 (4): 332.

Stokes, P.D. 2006. Creativity from constraints: The psychology of break-

through. New York: Springer Publishing Co.

Thompson, G., and M. Lordan. 1999. Review of creativity principles

applied to engineering design. Proceedings of the Institution of Mechanical

Engineers, Part E: Journal of Process Mechanical Engineering 213 (1): 17–31.

Torrance, E.P. 1974. Torrance tests of creative thinking. Lexington, MA:

Personnel Press/Ginn and Co./Xerox Education Co.

Treffinger, D.J. 2003. Assessment and measurement in creativity and

creative problem solving. In The educational psychology of creativity, ed. J.C.

Houtz. Cresskill, NJ: Hampton Press.

Weisberg, R. 1986. Creativity: Genius and other myths. New York:

W.H. Freeman/Times Books/ Henry Holt & Co.

Weisberg, R.W. 1999. Creativity and knowledge: A challenge to theories.

New York: Cambridge University Press.

April 2009 Journal of Engineering Education 153

AUTHORS’ BIOGRAPHIES

Christine Charyton was previously a visiting assistant professor

in clinical/counseling psychology at Ohio State University, Newark

and is currently a lecturer in the Department of Psychology at Ohio

State University and a psychologist at the Cognitive Behavioral

Center of Greater Columbus. Dr. Charyton studies the aesthetic

science of creativity and innovation in engineering and music. Her

research emphasizes the necessity of creativity as a vehicle for inno-

vation in engineering education and industry. Her dissertation in-

vestigated general, artistic, and scientific creativity and risk toler-

ance in engineering and music students. Dr. Charyton also

conducts research using nonlinear dynamics and studies neurologi-

cal conditions such as epilepsy. She provides psychotherapy to chil-

dren, adolescents, and adults with a variety of psychological condi-

tions. She has postdoctoral expertise in treating children with

autism, pervasive developmental disorders, ADHD and other

conditions and has co-facilitated parenting groups.

Address: Ohio State University, 1827 Neil Avenue, 130 Lazenby

Hall, Columbus, OH 43210; e-mail: charyton.1@osu.edu. Contact

the authors by e-mail for additional detailed material.

John A. Merrill is the director for the First-Year Engineering

Program at The Ohio State University College of Engineering,

and has worked in this capacity for over nine years. The program

serves approximately 1,300 students annually in courses orga-

nized to ensure student success through rigorous academics in a

team-based environment. His responsibilities include opera-

tions, faculty recruiting, curriculum management, student reten-

tion, and program assessment. Dr. Merrill received his Ph.D. in

Instructional Design and Technology from The Ohio State

University in 1985, and has an extensive background in public

education, corporate training, and contract research. He has

made frequent presentations at conferences held by the Ameri-

can Society for Engineering Education (ASEE) and its affiliate

conference, Frontiers in Education (FIE). He is part of the re-

search team that was recently awarded an NSF grant to study

strategies for maximizing success among students with learning

disabilities.

Address: The Ohio State University, 2070 Neil Ave., 244C

Hitchcock Hall, Columbus, OH 43210-1278; telephone: (�1)

614.292.0650; fax: (�1) 614.247.6255; e-mail: merrill.25@

osu.edu.

154 Journal of Engineering Education April 2009

APPENDIX

April 2009 Journal of Engineering Education 155

Appendix A. CEDA Sample Problem (see also Charyton, Jagacinski, and Merrill, 2008).

156 Journal of Engineering Education April 2009

Appendix B. Scoring the CEDA. (see also Charyton, Jagacinski, and Merrill, 2008).