Abstract In this chapter contribution to the edited volume, Branco Ponomariov and Craig Boardman explore the potential usefulness of a standardized assessment of center impacts relevant to all stakeholder groups by focusing on knowledge trans-actions and their organizational outcomes, rather than exclusively on the production of discrete, stakeholder-specific outcomes (e.g., publications, patents, processes, products, human capital). Using survey data, the authors analyze the relationship of knowledge transactions (e.g., type, frequency, duration, content, usage, formalness) to other general stakeholder outcomes (e.g., improvement in organizational capac-ity). Ponomariov and Boardman discuss their focus as a potentially fruitful approach to center evaluation insofar that it speaks directly to the rationale of the major cen-ters programs both in the United States and abroad: to advance collective capacity towards dealing with complex societal problems through boundary-spanning col-laboration. For complementary examinations, see the chapters on the challenges to evaluating government cooperative research centers programs by Irwin Feller and colleagues and by David Roessner and colleagues.
B. Ponomariov (*) Department of Public Administration , University of Texas at San Antonio , 501 West Cesar Chavez Boulevard , San Antonio , TX 78207 , USA e-mail: [email protected]
C. Boardman John Glenn School of Public Affairs, Battelle Center for Science and Technology Policy , The Ohio State University , Columbus , OH 43210 , USA e-mail: [email protected]
Chapter 3 Does Industry Bene fi t from Cooperative Research Centers More Than Other Stakeholders? An Exploratory Analysis of Knowledge Transactions in University Research Centers
Branco Ponomariov and Craig Boardman
60 B. Ponomariov and C. Boardman
3.1 Introduction
Much of the now “classic” literature on cooperative research centers examines bene fi ts for industrial companies participating in university research centers (a speci fi c type of cooperative research center, as discussed in the Introduction to this volume by Gray, Boardman, and Rivers), such as the conduct of applied and commercially rel-evant research by university faculty (Gray et al. 2002) and access to upstream modes of knowledge and to students for hire upon graduation (Feller et al. 2002 ) , among other bene fi ts (Feller et al. 2002 ; Gray and Steenhuis 2003 ; Roessner et al. 1998 ) . But there remain a number of important unaddressed questions regarding industry bene fi t from cooperative research centers.
Other chapters in this volume extend the “industry bene fi t” component of the centers literature by addressing why private companies participate in university research centers (see the chapter contribution from Hayton, Sehili, and Scarpello) and by outlining the problems of measuring the economic impacts of centers (see the chapter contribution from Roessner, Manrique, and Park). The current chapter extends understanding of industry bene fi t from centers by reporting on a relatively broad range of industry outcomes and comparing these outcomes to those gained by other types of center stakeholders, including government agencies, other public organizations like public utilities and government research laboratories, and by universities.
The impetus for this exploratory investigation is the apparent bifurcation in the university research centers literature focused on university–industry relations (Bozeman and Boardman forthcoming). On one hand, there are the numerous studies similar to this and the other chapters in this edited volume focused on industry bene fi t from centers. On the other hand, there is what in our view is the rather alarmist component of the literature characterizing university–industry interactions facilitated by university research centers as disruptive and potentially harmful to nonindustry stakeholders. For instance, the “academic capitalism” lit-erature (e.g., Slaughter and Rhoades 1996, 2004 ; Slaughter and Leslie 1997 ; Slaughter et al. 2002 ) implies, sometimes suggests explicitly, that university research centers detract from the public service missions and educational goals of the academy.
A fi rst step towards addressing the bene fi ts afforded industry vis-à-vis other cen-ter stakeholders is to monitor and compare the center-related transactions of these stakeholders. By transactions, we mean the time, duration, and frequency of differ-ent knowledge development and knowledge transmission events (though due to data limitations we can only address some of these currently). These may include infor-mal contacts, presentations, accessing information available at the center Web site, center reports, products usage, etc. We start with a focus on transactions rather than on outcomes per se because the former are in a sense standardized outcomes that allow for a valid cross-stakeholder comparison. Next, these transactions are related descriptively to general outcomes, including stakeholder indication of a knowledge
613 Does Industry Bene fi t from Cooperative Research Centers…
contribution to a speci fi c project, stakeholder indication of a knowledge contribu-tion leading to organizational change, and so on. Like our initial focus on transactions, our operationalization of outcomes emphasizes the general rather than the speci fi c, due to our goal of comparing validly industry bene fi ts to the bene fi ts gained by other center stakeholders.
Because this study is exploratory, we do not frame the analysis with formal theo-ries or hypotheses. However, our past experiences evaluating university research centers for the US National Science Foundation (NSF) and the US National Institutes of Health (NIH), as well as our own intuition based on the historic development of university research centers programs at the Federal level (e.g., with as much focus on education as on industry bene fi t, see Suh 1986), suggest that the fi ndings will reveal that while industry indeed bene fi ts from center participation, so too do other stakeholders bene fi t. Therefore, informally, we do not expect to see signi fi cant dif-ferences in the transactions nor in the general outcomes between industry stake-holders and other types of center stakeholder. Instead, we expect to see broad-based center contributions to a wide range of stakeholder groups, including but not limited to industry. Though our thinking may defy some past fi ndings focused on industry bene fi t, we feel that these studies fi nd such differences due to the industry-speci fi c nature of the outcome measures used (e.g., spin-off fi rms, patents, licensing), and not because universities and governments bene fi t less than industry from center participation.
To address our intuition empirically, this chapter uses survey data from a long-established university research center—the Mid-America Earthquake (MAE) Center, an NSF Engineering Research Center established in 1997 at the University of Illinois at Urbana-Champaign. The MAE Center provides an excel-lent opportunity for an initial investigation of comparative transactions and out-comes between industry and other center stakeholders because it has reached successfully the conclusion of its funding cycle with the NSF and therefore has had time for industry and other stakeholders to experience and re fl ect upon their respective center transactions and outcomes. Additionally, the MAE Center is a good case for developing a better understanding of how centers may (or may not) affect different stakeholders differently, because the MAE Center is part of what many consider to be the fl agship centers program in the United States (Bozeman and Boardman 2004). The MAE Center as a case study is therefore of “instru-mental” value (Yin 2003 ) .
This chapter is organized as follows. First, we provide background information on the NSF Earthquake Engineering Research Centers (EERCs) in the broader US science policy context to illustrate the inherent expectation behind many center pro-grams for substantial public and private (and not just industry) stakeholder partici-pation. Next, we provide an overview of MAE Center stakeholders and then compare a range of “knowledge transaction outcomes” across the stakeholder groups, includ-ing but not limited to industrial stakeholders, concluding with a discussion of the fi ndings and of the implications for future evaluation and research.
62 B. Ponomariov and C. Boardman
3.2 Case Background: NSF Earthquake Engineering Research Centers and the Stakeholder Composition of the MAE Center
The MAE Center is one of the three NSF-funded EERCs. The area of earthquake mitigation is an informative and tractable example of a societal goal addressed by means of creating designated boundary-spanning organization to explore the atten-dant technological options in cooperation with all relevant stakeholders—true to the original rationale for the center mechanism in the US science policy (Bozeman and Boardman 2004).
The goal of reducing the social and economic impacts of earthquakes is a good example of “system level” technologies that EERCs are intended to develop, insofar that it does not entail pursuing a discrete technological breakthrough, but rather the improvement of the system of actors and technologies involved in planning for and mitigating the consequences of earthquakes. This necessitates the integration of multiple disciplines and streams of knowledge, including civil engineering, com-puter science, the social sciences, etc. Predicting and mitigating the behavior of structures and socio-technical systems exposed to earthquakes is of interest to a broad swath of stakeholders, including private civil engineering and architectural fi rms, materials suppliers, insurance companies, as well as federal, state, and local government agencies. Therefore, a center devoted to systemic understanding and mitigation of earthquake hazards addresses the knowledge needs of multiple and heterogeneous actors, industry and nonindustry, which is precisely the intent codi fi ed in the Earthquake Hazards Reduction Act of 1977 (Public Law 95-124, 42 U.S.C. 7701 et. seq.). 1
The NSF’s mandate as explicitly set forth within the 1977 Act is to fund research “on earth science to improve the understanding of the causes and behavior of earth-quakes, on earthquake engineering, and on human response to earthquakes,” in part by means of supporting “university research consortia and centers for research in geosciences and in earthquake engineering” to work in conjunction with relevant agencies (e.g., the United States Geological Survey [USGS], the National Institute of Standards and Technology [NIST], the Federal Emergency Management Administration [FEMA]), as well as state and local governments, and industry stakeholders and private companies.
To meet these objectives, the NSF funded three EERCs. As earthquakes are cata-lysts for numerous decisions regarding emergency planning and response, it is inte-gral to understand how the knowledge produced by these centers is used by stakeholders including those not just in industry, but additionally in local and state government as well as academia. Of particular interest for the purpose of this chap-ter (as well as for the NSF) is understanding whether the knowledge produced by the MAE Center and its counterparts is useful across an interconnected and diverse web of both public and private stakeholders.
1 As amended by Public Laws 101614, 105-47, 106-503, and 108-360.
633 Does Industry Bene fi t from Cooperative Research Centers…
The stakeholder composition of the MAE Center is diverse and includes a wide range of annual dues-paying “members” interested in managing or mitigating the consequences of earthquakes, including government agencies, public utility compa-nies, private engineering fi rms, and public and private infrastructure owners. In addition, private sector companies may also interact with the MAE Center as “practitioners” who can purchase access to center products and services, or enter contract research agreements separately. Table 3.1 summarizes the core stakehold-ers of the MAE Center.
3.3 Data and Method: Survey-Based Case Study of the MAE Center
We use a stakeholder survey to make basic statistical comparisons of the variable center–stakeholder knowledge interactions and outcomes, by stakeholder type. Though we de fi ne stakeholders more broadly here than just industrial members, analytically it is important to evaluate just those stakeholders that have had more than a passing interaction with the MAE Center. In preparation for the stakeholder survey, preliminary stakeholder interviews were conducted at the 2006 MAE Center Annual Meeting, held in Austin, Texas. The interviews were used to develop a list of “involved” MAE Center stakeholders—conjointly with MAE Center staff and both current and past members of the Center’s Executive Advisory Board, Industrial Stakeholder Advisory Board, as well as individuals identi fi ed by MAE Center prin-cipal investigators as playing a key role in active projects. The result of this effort was a survey mailing list of 80 individuals representing 52 organizations involved with the MAE Center.
A Web-based survey was administered to this group in August 2006. Overall, the survey was completed by 51 respondents, from a range of stakeholders including but not limited to industrial stakeholders for a response rate of over 63%. Only fi ve of the stakeholder respondents to the survey were directly involved in the initial establishment of the MAE Center. All other respondents were either recruited by the MAE Center to join as members or to participate or became af fi liated as a result of a professional or informal interaction. (For a closer look at how centers market to and recruit new members, see the Rivers and Gray chapter in this volume.) See Fig. 3.1 for a summary of the survey respondents.
Figure 3.1 shows the MAE center stakeholder composition to be diverse, re fl ecting substantial participation from all stakeholder groups with signi fi cant con-cern about earthquakes and their social and economic consequences. Even though at the organizational level industrial companies represent the predominance of stake-holders (see Table 3.1 ), at the individual level Fig. 3.1 shows industrial stakeholders not to be the largest group; local and Federal government representatives represent the largest share of stakeholders. However, the diversity of stakeholders does not itself speak to the relative bene fi ts these stakeholders receive from participating in the MAE Center. We next assess the relative frequency and signi fi cance of knowledge
64 B. Ponomariov and C. Boardman
Tabl
e 3.
1 St
akeh
olde
r co
mpo
sitio
n of
the
MA
E C
ente
r (o
rgan
izat
iona
l lev
el o
f an
alys
is)
Mem
bers
Se
ctor
/indu
stry
Pr
actit
ione
rs
Sect
or/in
dust
ry
Fede
ral H
ighw
ay A
dmin
istr
atio
n G
over
nmen
t age
ncy
AB
S G
roup
, Inc
. Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
Il
linoi
s E
mer
genc
y M
anag
emen
t Age
ncy
Gov
ernm
ent a
genc
y A
mer
ican
Fam
ily I
nsur
ance
Pr
ivat
e se
ctor
/insu
ranc
e fi r
m
Mar
riot
t Int
erna
tiona
l Pr
ivat
e se
ctor
/infr
astr
uctu
re
Am
eric
an I
nstit
ute
of S
teel
Con
stru
ctio
n Pr
ivat
e se
ctor
/inte
rest
gro
up
NO
AA
Coa
stal
Ser
vice
s C
ente
r G
over
nmen
t age
ncy
Am
eric
an R
e-In
sura
nce
Com
pany
Pr
ivat
e se
ctor
/insu
ranc
e fi r
m
Paci
fi c G
as &
Ele
ctri
c Pr
ivat
e se
ctor
/util
ity c
ompa
ny
Aon
Cor
pora
tion
Priv
ate
sect
or/in
sura
nce
fi rm
Sk
idm
ore,
Ow
ings
& M
erri
ll Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
B
owm
an, B
arre
tt &
Ass
ocia
tes,
Inc
. Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
T
he W
orld
Ban
k B
rick
Ind
ustr
y A
ssoc
iatio
n Pr
ivat
e se
ctor
/inte
rest
gro
up
Con
stru
ctio
n Te
chno
logy
Lab
orat
orie
s, I
nc.
Priv
ate
sect
or/e
ngin
eeri
ng fi
rm
Ear
thqu
ake
Haz
ards
Sol
utio
ns
Priv
ate
sect
or/e
ngin
eeri
ng fi
rm
Fede
ral H
ighw
ay A
dmin
istr
atio
n G
over
nmen
t age
ncy
Geo
mat
rix
Con
sulta
nts,
Inc
. Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
IB
HS
Priv
ate
sect
or/r
esea
rch
lab
Kin
emet
rics
, Inc
. Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
R
isk
Man
agem
ent S
olut
ions
, Inc
. Pr
ivat
e se
ctor
/insu
ranc
e fi r
m
Serv
drup
Civ
il, I
nc.
Priv
ate
sect
or/e
ngin
eeri
ng fi
rm
Sieb
old
Sydo
w E
lfan
baum
Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
St
ate
Farm
Ins
uran
ce
Priv
ate
sect
or/in
sura
nce
fi rm
U
S A
rmy
CE
RL
G
over
nmen
t age
ncy/
rese
arch
labo
rato
ry
Will
mer
Eng
inee
ring
, Inc
. Pr
ivat
e se
ctor
/eng
inee
ring
fi rm
653 Does Industry Bene fi t from Cooperative Research Centers…
transactions and outcomes across stakeholder type. We emphasize “knowledge transactions” rather than discrete outcomes due to the need to compare stakeholder bene fi ts in a standardized way. Though industrial and other stakeholders may bene fi t from the MAE Center in quite distinct ways, these disparate bene fi ts are arguably the result of knowledge transactions, per the NSF mission for its EERCs.
The survey operationalizes what we call “knowledge transactions” by asking respondents about the frequency, duration, and perceptions of their respective inter-actions with the MAE Center. The methods of analysis are descriptive. The popula-tion and sample of interest are very small, and furthermore, the key “independent variables” (i.e., the different stakeholder groups) are nominal, which does not allow for robust statistical analysis (although c 2 and p -values are provided for reference). Our general expectation, as outlined in the introduction, is that there in fact should be no striking differences in stakeholder outcomes and interactions across different stakeholder groups; any observed differences in the descriptive statistics below are interpreted substantively, and the limitations of the analysis are discussed in the conclusion.
3.4 Findings: Comparing MAE Center Knowledge Transactions for Industry Stakeholders to the Knowledge Transactions of Nonindustry Stakeholders
First, we examine usefulness and frequency of knowledge transactions by stakeholder type. Though we expect all MAE Center stakeholders to be involved in and bene fi t from knowledge transactions related to the Center, including but not limited to industrial stakeholders, different stakeholder groups may have interactions with the Center that vary in substantive content as well as experientially (e.g., in terms of frequency, perceived usefulness). Substantively or in terms of the content of trans-actions, engineering fi rms may be interested primarily in learning about knowledge advances in earthquake-resistant building technology, while insurance companies and local governments may be more interested in learning about new software
10%
33%
43%
Local, state or federalgovernment
Private Sector
Research or educationalorganization
Public utility company
14%
Fig. 3.1 Composition of stakeholder survey respondents by af fi liation
66 B. Ponomariov and C. Boardman
methods and applications for earthquake prediction and response. Experientially, different stakeholders may access different knowledge sources within the MAE Center, ranging from the Center webpage to reports and articles to person-to-person interactions with Center faculty and staff. Because of our interest in making stan-dardized comparisons between industry and other types of stakeholders, we focus here on experiential differences, starting with the perceived usefulness of MAE Center knowledge transactions, by stakeholder type.
Table 3.2 displays the percentage differences and mean scores in the perceived usefulness of different sources/channels of MAE Center knowledge. Consistent with our expectations, none of the observable differences are statistically signi fi cant, though it is possible that government and industry stakeholders may value interac-tions with MAE Center staff/researchers marginally higher than do research organi-zations and public utilities—likely because research organizations have greater internal research capacity relative to fi rms and agencies. Similar, the results in Table 3.2 suggest that industry stakeholders may rank the value of center reports marginally higher than rank any other stakeholder group, and also that industry stakeholders may be more dependent on the internal knowledge capacity/staff of the MAE Center when compared to the other stakeholder groups.
Next, we examine the results for the set of survey items asking respondents to identify the frequency of use and usefulness of a variety of Center and non-Center information sources. Descriptive analysis of the reports suggests four possible com-posite variables: frequency of use of MAE Center-related sources, perceived useful-ness of MAE Center-related sources, frequency of use of other information resources (e.g., consulting fi rms, library and internet searches, professional association), and perceived usefulness of these alternate information sources. Speci fi cally, a factor analysis with varimax rotation and a variable-factor correlation threshold of 0.3 vali-dates these composite measures, by extracting four distinct factors. The four summa-tive scales exhibit acceptable internal validity (Cronbach’s a between 0.78 and 0.8). Consistent with our general expectation that perceived value of the center products and knowledge transactions at the aggregate is likely to be uniform across stake-holder groups, there are no statistically signi fi cant differences in these summary variables across the groups, thus we do not report the individual results separately.
Table 3.3 reports the correlations among the four factors. There is no statistically signi fi cant correlation between the frequency of using MAE Center sources and their perceived usefulness, nor between the perceived usefulness of alternate vs. Center sources, though the frequency of using alternate sources is correlated signi fi cantly with the frequency of use of Center sources (Corr = 0.37, p = 0.01). However, frequency of use of Center sources is correlated with perceived usefulness of alternate sources (Corr = 0.33, p = 0.03), and the frequency of use and usefulness of alternate resources are also correlated (Corr = 0.35, p = 0.02). Combined, these results suggest a mediating role for Center knowledge sources for stakeholders; the value of the MAE Center sources of hazards knowledge is contingent on broad information-seeking behaviors that utilize both generic and Center related sources. Rather than provide discrete or standalone value to stakeholders, MAE Center knowledge sources may be enhancing absorptive capacity (see the correlation between frequency of using Center sources and the perceived usefulness of
673 Does Industry Bene fi t from Cooperative Research Centers…
alternate sources). Collectively these results suggest that while the bene fi ts and value of MAE Center knowledge transactions may apply to all stakeholder groups, it is possible that they materialize through different mechanisms. Next, we focus more directly on the bene fi ts that MAE Center knowledge transactions may afford different stakeholders.
Table 3.2 Percentage responses to “Overall, how useful have you found these [MAE Center] resources?” ( N = 51)
Public utility (%) Government (%)
Private sector (%)
Research org. (%) Total (%)
MAE Center staff/researchers a Very useful 33.3 52.4 50.0 42.9 48.9 Somewhat useful 66.7 33.3 43.8 28.6 38.3 I have never used this 0.0 14.3 6.3 28.6 12.8 Mean (never used responses
excluded) 1.7 1.4 1.5 1.4 1.4
MAE Center reports b Very useful 25.0 28.6 43.8 28.6 33.3 Somewhat useful 75.0 47.6 50.0 42.9 50.0 Not useful at all 0.0 4.8 0.0 0.0 2.1 I have never used this 0.0 19.1 6.3 28.6 14.6 Mean (never used responses
excluded) 1.8 1.7 1.5 1.6 1.6
MAE Center webpage c Very useful 25.0 14.3 18.8 28.6 18.8 Somewhat useful 25.0 57.1 43.8 28.6 45.8 Not useful at all 0.0 4.8 12.5 0.0 6.3 I have never used this 50.0 23.8 25.0 42.9 29.2 Mean (never used responses
excluded) 1.5 1.9 1.9 1.5 1.8
MAE Center annual meetings d Very useful 33.3 14.3 31.3 42.9 25.5 Somewhat useful 66.7 42.9 31.3 28.6 38.3 Not useful at all 0.0 4.8 18.8 0.0 8.5 I have never used this 0.0 38.1 18.8 28.6 27.7 Mean (never used responses
excluded) 1.7 1.8 1.8 1.4 1.8
Internal research staff in your organization e Very useful 0 41 65 43 45 Somewhat useful 40 27 18 29 25.6 I have never used this 60 32 18 29 29 Mean (never used responses
excluded) 2 1.4 1.2 1.4 1.4
a c 2 (6) = 3.6, Pr = 0.7 b c 2 (9) = 5.25, Pr = 0.8 c c 2 (9) = 5.2, Pr = 0.8 d c 2 (9) = 8.6, Pr = 0.5 e c 2 (9) = 7.1, Pr = 0.3
68 B. Ponomariov and C. Boardman
Table 3.4 shows that stakeholders did not differ much in terms of the MAE Center leading to organizational changes (e.g., in operations or processes) or to improve-ment in understanding of seismic hazards (though there are no differences across the groups, the majority of stakeholders agree to have experienced such bene fi t). However, per the observed percentage differences it is possible that for industry stakeholders, information provided by the MAE Center to have been more valuable relative to other stakeholder groups (86% vs. 72% for the stakeholder population as a whole).
Tables 3.5 , 3.6 , and 3.7 are intended to uncover differential mechanisms via which different stakeholder groups may experience Center-related bene fi ts—by cross-tabulating the frequency of knowledge transactions in Table 3.2 with stake-holder outcomes reported in Table 3.4 , for all stakeholders (Table 3.5 ), and then for the industry (Table 3.6 ) and nonindustry (Table 3.7 ) subgroups. The purpose is to provide support for the conceptual claim advanced here, namely that speci fi c knowl-edge transactions or events can be potentially related to speci fi c stakeholder out-comes. Although granular data on speci fi c interactions and events is not viable to collect with survey instruments, overall relationships may be discerned. Ideally, an
Table 3.4 Percentage responses for different types of perceived stakeholder outcomes ( N = 51), by stakeholder type
Public utility (%) Government (%) Private sector (%) Research org. (%) Total (%)
The MAE Center has increased my organization’s understanding of seismic hazards a No Yes
66.7 33.3
40 60
31 69
29 71
37 63
There have been concrete changes in operations or processes as a result of information gained from the MAE Center b
No 66.7 64.7 75 66.7 68.4 Yes 33.3 35 25 33.3 31.6 Information provided by the MAE Center has been considered valuable by others in my organization c No 50 28 14 50 27 Yes 50 72 86 50 72
a c 2 (9) = 9.7, Pr = 0.37 b c 2 (9) = 10.2, Pr = 0.34 c c 2 (6) = 5.9617, Pr = 0.43
Table 3.3 Correlations between the indices for center-related and generic information sources frequency of use and perceived usefulness ( N = 51)
693 Does Industry Bene fi t from Cooperative Research Centers…
Tabl
e 3.
5 C
orre
latio
ns b
etw
een
freq
uenc
y of
use
of
MA
E k
now
ledg
e tr
ansf
er c
hann
els
and
stak
ehol
der
outc
omes
for
all
stak
ehol
ders
( N
= 5
1)
MA
E h
as in
crea
sed
by u
nder
stan
ding
of
sei
smic
haz
ards
Info
rmat
ion
prov
ided
by
MA
E h
as b
een
cons
ider
ed
valu
able
by
othe
rs in
my
orga
niza
tion
Mad
e co
ncre
te c
hang
es in
ou
r op
erat
ions
or
proc
esse
s as
a r
esul
t of
info
rmat
ion
gain
ed f
rom
MA
E
MA
E
web
page
M
AE
re
port
s M
AE
sta
ff/
rese
arch
ers
Info
rmat
ion
prov
ided
by
MA
E
cons
ider
ed v
alua
ble
by
othe
rs in
my
org.
0.55
***
Mad
e co
ncre
te c
hang
es in
our
op
erat
ions
or
proc
esse
s 0.
54**
* 0.
60**
*
MA
E w
ebpa
ge
0.29
**
0.36
**
0.19
M
AE
rep
orts
0.
42**
* 0.
31*
0.24
* 0.
66**
* M
AE
sta
ff/r
esea
rche
rs
0.06
0.
19
0.04
0.
42**
* 0.
64**
* M
AE
ann
ual m
eetin
gs
0.37
**
0.37
* 0.
30**
0.
26**
0.
48**
* 0.
03**
* p £
0.1
**
p £
0.5
*** p
£ 0
.01
70 B. Ponomariov and C. Boardman
Tabl
e 3.
6 C
orre
latio
ns b
etw
een
freq
uenc
y of
use
of
MA
E k
now
ledg
e tr
ansf
er c
hann
els
and
stak
ehol
der
outc
omes
for
all
indu
stry
sta
keho
lder
s ( N
= 1
7)
Info
rmat
ion
prov
ided
by
MA
E h
as b
een
cons
ider
ed
valu
able
by
othe
rs in
my
orga
niza
tion
MA
E h
as in
crea
sed
by
unde
rsta
ndin
g of
sei
smic
ha
zard
s
Mad
e co
ncre
te c
hang
es
in o
ur o
pera
tions
or
proc
esse
s M
AE
w
ebpa
ge
MA
E
repo
rts
MA
E s
taff
/re
sear
cher
s
MA
E h
as in
crea
sed
by u
nder
stan
ding
of
sei
smic
haz
ards
0.
68**
*
Mad
e co
ncre
te c
hang
es in
our
op
erat
ions
or
proc
esse
s 0.
00
0.53
*
MA
E w
ebpa
ge
0.61
**
0.37
0.
27
MA
E r
epor
ts
0.24
0.
60*
0.40
0.
5*
MA
E s
taff
/res
earc
hers
0.
00
0.12
0.
29
0.26
0.
61**
M
AE
ann
ual m
eetin
gs
0.33
0.
36
−0.
26
0.16
0.
36*
0.07
* p £
0.1
**
p £
0.5
*** p
£ 0
.01
713 Does Industry Bene fi t from Cooperative Research Centers…
Tabl
e 3.
7 C
orre
latio
ns b
etw
een
freq
uenc
y of
use
of
MA
E k
now
ledg
e tr
ansf
er c
hann
els
and
stak
ehol
der
outc
omes
for
non
indu
stry
sta
keho
lder
s ( N
= 3
4)
Info
rmat
ion
prov
ided
by
MA
E h
as b
een
cons
ider
ed
valu
able
by
othe
rs in
my
orga
niza
tion
MA
E h
as in
crea
sed
by
unde
rsta
ndin
g of
sei
smic
ha
zard
s
Mad
e co
ncre
te c
hang
es
in o
ur o
pera
tions
or
proc
esse
s M
AE
w
ebpa
ge
MA
E
repo
rts
MA
E s
taff
/re
sear
cher
s
MA
E h
as in
crea
sed
by u
nder
stan
ding
of
seis
mic
haz
ards
0.
57**
*
Mad
e co
ncre
te c
hang
es in
our
op
erat
ions
or
proc
esse
s 0.
81**
* 0.
62**
*
MA
E w
ebpa
ge
0.29
0.
31
0.2
MA
E r
epor
ts
0.29
0.
39*
0.27
0.
77**
* M
AE
sta
ff/r
esea
rche
rs
0.26
0.
18
0.2
0.6*
**
0.66
***
MA
E a
nnua
l mee
tings
0.
25
0.3*
0.
4**
0.34
**
0.5*
**
0.41
**
* p £
0.1
**
p £
0.5
*** p
£ 0
.01
72 B. Ponomariov and C. Boardman
analysis such as this should examine the presence and nature of such relationships for each stakeholder group, but this is not possible currently due to the limited num-ber of observations for the MAE Center case study.
Starting with the general fi ndings across all stakeholders (Table 3.5 ), it appears that there are some complementarities between some knowledge transfer channels or events. For example, annual meetings and reports are correlated signi fi cantly, which perhaps is not so surprising given that oftentimes for NSF centers including but not limited to EERCs annual reports are distributed at annual meetings. Use of MAE Center reports is most strongly correlated with frequency of interacting with MAE researchers, which may be indicative of relational intensity and knowledge transfor-mation. There is virtually no correlation between formal (e.g., annual meetings) and informal (e.g., person-to-person interactions) knowledge transfer channels.
There are also discernible correlations between perceived MAE Center-produced information usefulness and the likelihood of implementing changes within stake-holder organizations. Further examining the results for all stakeholders (Table 3.5 ), perceived MAE contribution to understanding of seismic hazards and higher valua-tion of MAE provided information by others in the stakeholder organization are strongly and signi fi cantly correlated with the likelihood of a change in operations or processes. No discernible correlations are found between making an organizational change and the frequency of interaction with MAE Center researchers or use of the MAE Center webpage.
Table 3.5 also shows some variance in the correlations between the frequency of usage of different knowledge transfer channels and the perceived value of the infor-mation and likelihood of implementing organizational changes. Speci fi cally, the frequency of using center reports are uniformly moderately correlated with increased understanding of earthquake hazards and likelihood of implementing organizational changes. MAE Center annual meetings may be similarly associated with these stakeholder outcomes, though perhaps less so.
Interesting given the importance of social capital in the extant knowledge trans-fer literature, Table 3.5 shows frequency of interacting with MAE Center staff and researchers not to be correlated with any of stakeholder outcomes. However, inter-actions still may be playing a mediating role because there is a strong relationship between interacting with MAE Center researchers and using MAE Center reports, as well as a moderate relationship between using MAE Center reports and imple-menting organizational change and generally valuing MAE Center-produced infor-mation; in combination, this is indirect evidence suggesting a complementary role between knowledge transfer and transformation—while informal interactions may not necessarily directly affect stakeholder outcomes, they appear to be inherently complementary of utilization of other knowledge transfer channels, e.g. actual knowledge outputs. However, regression analysis (excluded here) does not support this intuition.
Only some of the initially registered correlations (Table 3.5 ) persist for the indus-trial stakeholder group alone (Table 3.6 ). The signi fi cant correlation between inter-acting with MAE Center researchers and the use of MAE Center reports remains, as does the signi fi cant correlation between MAE Center annual meetings and the use
733 Does Industry Bene fi t from Cooperative Research Centers…
of the MAE Center webpage with the frequency of use of MAE Center reports. In turn, use of MAE Center reports is correlated with increased understanding of seismic hazards, which is one of the two variables besides usage of the MAE Center webpage to be correlated with overall value of MAE Center-provided information to the stakeholder organization. For industry stakeholders, there is only one variable signi fi cantly correlated with likelihood of implementing organizational or proce-dural changes as a result of MAE Center-provided information: increased under-standing of seismic hazards.
Not inconsistent with our general expectation that center ties will afford bene fi ts both to industry and nonindustry stakeholders, the correlations for nonindustrial stakeholders (Table 3.7 ) render results virtually identical with the correlation matrix for all (Table 3.5 ), and with a few exceptions. However, this also means that indus-try and nonindustry participants bene fi t differently, even when looking at outcomes that are standard enough for a valid comparison across stakeholder groups like knowledge transactions. Speci fi cally, the paths towards better understanding of seismic hazards seem more varied for nonindustry than for industrial stakeholders.
3.5 Conclusion: Towards a Knowledge Transactions Approach to Evaluating Cooperative Research Centers
This chapter compared knowledge transactions and outcomes across the MAE Center’s industry and nonindustry stakeholders. We began this comparison with the informal proposition that industry and nonindustry stakeholders alike engage in and gain from MAE Center-based knowledge transactions. This approach was based on our own intuition having worked as evaluators for university research centers—a particular type of cooperative research center—in response to the mostly anecdotal argument that such centers bene fi t industry more than other stakeholders. While indeed the basic descriptive analyses presented in the sections above show knowl-edge transactions and outcomes to be similar across stakeholder groups, the speci fi c relationships between knowledge transactions and outcomes seem to be mediated differently within the industry and nonindustry stakeholder groups.
The latter result serves the second purpose of this chapter—which is to begin articulating a rationale for conceptualizing center–stakeholder interactions as an ongoing accumulation of knowledge transactions. Though we have provided just a rudimentary description of such transactions using limited survey data for a single NSF center, the most important, and dif fi cult, question to address in the future with better data is whether it is possible to relate the nature and extent of different types of knowledge transactions to tangible stakeholder consequences at the project and organizational levels, across a wide variety of stakeholders, including but not lim-ited to industry. As we see it, the primary reason why the measurement of knowl-edge transactions and their outcomes has lagged is simply that there are no good measures for knowledge content and knowledge utilization, thereby skewing much of the evaluation and research of cooperative research centers towards better known
74 B. Ponomariov and C. Boardman
(and less actionable) knowledge outcomes such as publication counts and citations, which are useful proxies for knowledge volume and perhaps quality, but not for stakeholder knowledge utilization .
The conceptualization of center–stakeholder interactions as an ongoing accumu-lation of knowledge transactions is important because the goals of cooperative research centers almost never involve bene fi ting any speci fi c stakeholder group, but rather to resolve complex, systemic, social problems with scienti fi c and technologi-cal content. Such problem areas are not de fi ned by a speci fi c and discrete techno-logical bottleneck, but rather by a socio-technical network (Callon 1987 ) of actors with different stakes in and connections to the scienti fi c and technological artifacts produced. When considering cooperative research center impacts, scholars, and evaluators need to consider the extent to which centers satisfy the broad knowledge and technology needs of the actors or stakeholders the center connects, which more broadly may be understood as affecting and potentially enhancing relationships between social and technical actors in the context of a speci fi c problem area.
Thus, any assessment of center outcomes needs to be comparative and in being so must account for the variety of stakeholders and the salience of different types of outcomes, channels of knowledge transfer, and knowledge transactions. While there has been some comparison in this vein, e.g., in the academic capitalism literature (e.g., Slaughter and Rhoades 1996, 2004 ; Slaughter and Leslie 1997 ; Slaughter et al. 2002 ) , the focus in this small literature on stakeholder-speci fi c outcomes that are not valid for cross-stakeholder comparison means that formative and summative cooperative research center evaluation and research still has a long way to go in terms of making useful and actionable comparisons across stakeholder groups, for the bene fi t of all stakeholder groups including but not limited to industry.
Of course, attempting to capture knowledge transactions and their outcomes is not a departure from existing approaches, but there are several modi fi cations that may alter the nature of some evaluation questions and conclusions. Theoretically, the process of knowledge transfer is typically relationally intensive (Perkmann and Walsh 2007 ) and to some degree, contingent on the characteristics of the underlying knowledge (Rossi 2010 ) . The most fundamental, yet under-appreciated, character-istic of university–industry knowledge transfer is that in reality any knowledge transfer from university to industry involves not just the transmission or transfer of deliverables (Mowery and Sampat 2006) , but also knowledge transformation (Faulkner and Senker 1994 ) . At present, the knowledge transformation and trans-mission mechanisms are not suf fi ciently differentiated in evaluation practice, and the importance of formal and informal mechanisms has not been suf fi ciently systematized.
Moving forward, properly recorded and aggregated transactional data (i.e., the number, type, duration, and frequency of different knowledge development or trans-mission events) can allow for meaningful conceptualization of fairly complex inter-organizational processes, and may relate such transactional data to a variety of knowledge-related outcomes (because knowledge transactions happen in context). The proposed approach does not represent a major departure from current evalua-tion practice, but the conceptualization of these measures as knowledge transactions
753 Does Industry Bene fi t from Cooperative Research Centers…
that partially describe the process of knowledge transformation (vs. transfer) speaks much more directly to more general cooperative research center mechanism. The approach would constitute a substantively important re fi nement of one of the domi-nant existing approaches, including but not limited to bibliometric analysis.
References
Bozeman B, Boardman C (forthcoming) Academic faculty working in university research centers: neither capitalism’s slaves nor teaching fugitives. J High Educ
Bozeman B, Boardman PC (2004) The NSF Engineering research centers and the uiversity-industry research revolution. The Journal of Technology Transfer 29(3-4):365–375
Callon M (1987) The sociology of an actor-network: the case of the electric vehicle. In: Callon M, Law J, Rip A (eds) Mapping the dynamics of science and technology. McMillan, London
Faulkner W, Senker J (1994) Making sense of diversity: public-private sector research linkage in three technologies. Res Policy 23(6):673–695
Feller I, Ailes CP, Roessner JD (2002) Impacts of research universities on technological innovation in industry: evidence from engineering research centers. Res Policy 31(3):457–474
Gray DO, Steenhuis HJ (2003) Quantifying the bene fi ts of participating in an industry university research center: an examination of research cost avoidance. Scientometrics 58(2):281–300
Mowery D, & Sampat BN (2006) Universities in National Innovation Systems. In J. Fagerberg, D. Mowery & R. Nelson (Eds.), The Oxford Handbook of Innovation (pp. 209-239). New York: Oxford University Press.
Perkmann M, Walsh K (2007) University–industry relationships and open innovation: towards a research agenda. Int J Manage Rev 9(4):259–280
Roessner D, Ailes CP, Feller I, Parker L (1998) How industry bene fi ts from NSF’s Engineering Research Centers. Res Technol Manage 41(5):40–44
Rossi F (2010) The governance of university-industry knowledge transfer. Eur J Innov Manage 13(2):155–171
Slaughter S, Leslie LL (1997) Academic capitalism: politics, policies, and the entrepreneurial university. Johns Hopkins University Press, Baltimore
Slaughter S, Rhoades G (1996) The emergence of a competitiveness research and development policy coalition and the commercialization of academic science and technology. Sci Technol Hum Val 21:303–339
Slaughter S, Rhoades G (2004) Academic capitalism and the new economy. Johns Hopkins University Press, Baltimore, MD
Slaughter S, Campbell T, Folleman MH, Morgan E (2002) The “Traf fi c” in graduate students: graduate students as tokens of exchange between academe and industry. Sci Technol Hum Val 27(2):282–313
Suh, N. P. (1986). The Concept and Goals of the Engineering Research Centers The New Engineering Research Centers: Purposes, Goals, and Expectations (pp. 37-43). Washington, DC: National Academy Press.
Yin RK (2003) Case study research: design and methods. Sage, Thousand Oaks, CA
