The Alzheimer’s Disease Research Network and the Uniform Data Set1

Douglas R. Austrom2, Ph.D., Betty Barrett3, Ph.D., Elizabeth Merck4, Pamela A. Posey4, D.B.A., Bert Painter5, Ramkrishnan V. Tenkasi6, Ph.D.

INTRODUCTION Alzheimer's disease (AD) is recognized as a public health crisis worldwide (IADRP, 2013). AD is a complex neurodegenerative disease and the leading cause of dementia among the elderly people (Evans et al., 1989). Currently, there are approximately five million AD cases in the United States and about 35 million cases worldwide (Alzheimer's Disease International, 2009.

The focus of this case study is on the Uniform Data Set (UDS), a longitudinal database on Alzheimer’s patients, and the 29 Alzheimer’s Disease Centers (ADC) that submit their data to the UDS and actively collaborate in the ongoing maintenance, development and research

utilization of the database. The ADCs are based in major medical institutions across the United States. They have a multi-decade track record of collaborative research and a networked and virtual approach to the scientific study of AD. The central coordinating mechanism for the ADCs and the UDS is the National Alzheimer’s Coordinating Center (NACC), which is located at the

University of Washington. The NACC coordinates data collection and supports collaborative research among the ADCs.

Interviews were conducted with individuals from the NACC, the NIA, and the ADCs about the virtual work processes, the challenges facing participants in these virtual environments, and about the coordination mechanisms used to resolve those problems in order to co-create

knowledge. These data were then analyzed to determine what coordinating mechanisms were used to deal with and overcome the barriers to knowledge development and dissemination

that exist in virtual organizations.

This research site and project is one of three research sites included in our study of virtual R&D. We used an exploratory case study methodology and a sociotechnical systems analysis

framework to assess the factors that contribute to the effectiveness virtual R&D projects. This case study focused on two main research questions:

1. What distinguishes effective deliberations and forums from those that are ineffective in

virtual R&D projects? 2. What are the major variances or knowledge barriers in virtual R&D projects?

1 This material is based upon work supported by the National Science Foundation under grant number NSF OCI 09-43237. Any

opinions, findings and conclusions or recommendations in th is material are those of the author(s) and do not necessarily

reflect the views of the National Science Foundation. 2 Indiana University 3 Massachusetts Institute of Technology 4 Sociotechnical Systems Roundtable 5 Modern Times Productions 6 Benedictine University

SCAN OF THE ALZHEIMER’S DISEASE RESEARCH NETWORK

National Institute of Aging and the Alzheimer’s Centers In 1984, leaders at the National Institute of Aging (NIA) recognized that Alzheimer’s Disease and related dementias were catastrophic diseases (Butler, 2005) and decided to create a national, interdisciplinary research program focused on the causes and the course of Alzheimer’s disease. With remarkable foresight, they believed that a network of centers “could create the necessary

infrastructure to promote longitudinal clinical-pathological studies; integrate basic and clinical research; standardize clinical assessment tools, methods, and clinical trials; and establish

national data banks to share resources for clinical, neuropathological, and genetic studies” (National Institute of Aging, 2005).

The NIA originally funded five Alzheimer’s Disease Research Centers (ADRC) in 1984 and five additional centers in 1985. In 1986 the centers collaborated to establish the Alzheimer’s Disease Patient Registries (ADPR) and the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). CERAD standardized the definition, assessment, and profile of AD and the criteria for the diagnosis of AD that are still widely used today.

The scientists and clinicians at these 10 ADRCs provided the foundation of a network that today reaches across the country and includes ADCs at 29 major medical institutions (see figure 1 for

the geographic distribution of these centers).

Figure 1. Geographic Distribution of the 29 NIA-Funded Alzheimer’s Disease Centers

The ADCs conduct basic research on the mechanisms of AD and translational research to improve diagnosis of the disease and improve care for AD patients. In addition, they support

the patients and their families with the actual diagnosis, medical management of the disease,

and information about the disease and available services and resources.

The ADCs are a multidisciplinary mix of clinicians, researchers, and administrators; for example, neuropathologists, neurologists, neuropsychologists, geriatricians, geriatric psychiatrists,

radiology and imaging technicians, nurse practitioners, research technicians, social workers, clinical psychologists, epidemiologists, biostatisticians, data base managers, and other

computer specialists. While the ADCs are required as a condition of funding to have five common cores -- Clinical, Neuropathology, Data, Education, and Administration -- this does not preclude them from establishing other core clinical or research areas such as neuroimaging and genetics.

National Centers and Initiatives In addition to the ADCs, the AD research network includes several NIA-funded centers: the

National Cell Repository for Alzheimer’s Disease located at Indiana University, the Alzheimer's Disease Education and Referral Center in Bethesda, Maryland, coordinates and the National

Alzheimer’s Coordinating Center located at the University of Washington. The history of these centers is intertwined with that of the ADCs.

In 1989, the NIA created the National Cell Repository for Alzheimer’s Disease (NCRAD), a genetics cell repository. NCRAD banks DNA and cells, builds a database of family histories and medical records, and provides researchers with cell lines and DNA samples from normal control subjects and people with well-documented AD. The U.S. Congress created the Alzheimer's Disease Education and Referral (ADEAR) Center in

1990 to compile and disseminate “current, comprehensive, unbiased” information concerning AD for health professionals, people with AD and their families, and the public (NIA, 2013). In

addition to providing answers to specific questions about AD, free publications about the disease -- diagnosis, related disorders, risk factors, treatment, caregiving tips, home safety tips—and training materials, the ADEAR Center can provide information on ongoing clinical trials and referrals to the ADCs and other local services. Collaboration on multi-site clinical trials for the cholinesterase inhibitor, tacrine, led to the

creation of the Alzheimer’s Disease Cooperative Study (ADCS) in 1991. The ADCS coordinates a consortium of ADCs and affiliated organizations that conduct large clinical trials on promising compounds designed to improve cognitive functioning, slow the rate of decline, delay the onset of the disease.

In 1997, the NIA established the Minimum Data Set (MDC), which was originally housed at Rush

Presbyterian Hospital in Chicago to compile data on patients and control subjects enrolled in the

ADCs. The MDS included brief clinical and demographic information on over 74,000 patients and control subjects enrolled in the ADCs. While still valuable, the data were assembled retrospectively and were primarily cross-sectional rather than longitudinal and minimal by design. Also, the ADCs did not follow a standardized protocol when these data were originally

collected, even though the data were reported to NACC in defined data elements and the missing

data rate was as high as 30%.

National Alzheimer’s Coordinating Center (NACC) and the Uniform Data Set (UDS) Key individuals at the NIA and in the larger AD research community recognized the need for a

common database with longitudinal data. They believed that this could be best accomplished by creating a coordinating center for the ADCs. National Alzheimer’s Coordinating Center

(NACC) was established at the University of Washington by the NIA in 1999 to facilitate collaborative research among the ADCs (NACC website, 2010). The MDS was transferred to the NACC and the first task of the center was to reduce the error rate in the MDS to less than two percent. In 2002, the MDS was expanded to include neuropathological data from autopsies. In collaboration with neuropathology researchers, a new data form was put together that linked clinical diagnoses and pathology findings.

In the early 2000’s, the NIA created an External Advisory Committee to make recommendations

on the future of ADC program. They recommended expanding and standardizing clinical data collected at the ADCs and stored at the NACC. The NIA formed an ADC Clinical Task Force to plan this expansion, define an expanded data set, and standardize clinical evaluation for all ADCs. The ADC Clinical Task Force, working closely with the NACC, met regularly for three years to define the content of the new Uniform Data Set (UDS), develop common data collection templates, and construct a relational database. The ADCs agreed upon the UDS in early 2005. The NACC piloted the standardized data collection templates in the summer of 2005 and the common protocols were formally implemented that September.

The ADCs enroll subjects in various ways: referrals from clinicians, self-referrals by patients themselves or concerned family members, active recruitment through community

organizations, and volunteers who wish to serve as healthy controls. Subjects visit the ADCs annually and are assessed by clinicians, neuro-psychologists, and other ADC research personnel.

As many as 18 standardized forms are used and 725 data points are collected at each visit including socio-demographics, family history, dementia history, neurological exam findings, functional status, neuropsychological test results, clinical diagnosis, and imaging tests.

ADC research assistants enter the data from these standardized forms in the ADC’s local database. Data managers at the ADCs monitor the quality of the local data before submitting it electronically to the NACC each month. Data base managers at the NACC also conduct data

checks on the data they receive from the Centers before adding it to the UDS. The NACC makes payments of $29,000 per year to the ADCs to cover the costs associated with submitting their data.

A primary goal of the UDS is to accumulate a large-scale pool of data which all qualified scientists can use to discover better diagnostic criteria and treatment options for patients suffering from AD and related neurodegenerative diseases. Table 1 provides a description of the data available from the NACC As of June 2013, there are 28,444 subjects in the UDS

longitudinal database, 74,397 subjects from the original Minimum Data Set, and 13,279

subjects from the Neuropatholgy Data Set.

Table 1. Description of the Data Available from the NACC as of June 20137

Minimum Data Set

(MDS) Uniform Data Set (UDS)

(LONGITUDINAL) Neuropathology

Data Set (NP)

Years covered 1984 - 2005 Sept. 2005 - present 1984 - present

Study subjects Enrollees followed at ADCs (with & without dementia)

Enrollees followed at ADCs (with or without dementia)

Subjects who died and underwent autopsy

Approx. # of subjects*

74,397 28,444 13,279

Approx. # of variables

67 725 85

Method of data collection

Mainly abstracted retroactively from ADC medical records

Collected prospectively by clinicians, neuro-psychologists, and other ADC research personnel, using up to 18 standardized forms at each visit.

Standardized neuropathology form, completed by neuropathologist

Time period covered for each subject

Mainly status on last ADC visit; some variables also capture initial-visit status

Initial visit and each annual follow-up visit, plus milestones such as death or dropout

Status of brain at autopsy

Topics covered

Demographics, cognitive status, clinical dementia diagnosis, selected clinical manifestations, comorbid conditions, MMSE score, vital status, primary neuropathological diagnosis (if died and had brain autopsy)

Socio-demographics on subject and informant, family history, dementia history, neurological exam findings, functional status, neuropsychological test results, clinical diagnosis, whether imaging testing done, ApoE genotype

Demographics, date of death, primary and secondary neuropathological diagnoses, presence/absence of neuropathological features of most major dementias, APOE genotype, brain weights

The NACC also has limited funds to support collaborative research projects that involve at least

three ADCs as well as research projects proposed by junior investigators that use the UDS or Neuropathology data.

7 NACC, 2013, https://www.alz.washington.edu/WEB/data-descript.html

Any researcher can request access to the NACC’s database for basic and translational research.

The requests are reviewed by a methods group to make sure that they are legitimate. NAC’s staff will also format data if needed, and occasionally give analytic support. The NACC received

more than 700 requests in 2010. Since its inception, the NACC has been instrumental in 590 publications as of June 2013:8 191 by external researchers using NACC data, 100 by NACC

personnel, 173 from NACC funded projects, 34 based on NACC-funded secondary analysis, 72 from NACC–ADC collaboration in external studies, and 20 that relied on indirect NACC support.

The UDS project and the NACC have received some acclaim in the popular press as an exemplar of research collaboration (Kolata, 2010).

Figure 2. Structure of the ADC Committees

While the central role of the NACC is enabling coordination and collaboration among the 29 Alzheimer’s centers through the ongoing maintenance, development and research utilization of the UDS, the NACC also organizes the face-to-face (F2F) meetings of the ADC core directors and ADC steering committees. These meetings are held twice a year in conjunction with the annual conferences of American Neurological Association and the American Academy of Neurology. The structure of the ADC committees is shown in Figure 2. The members of the executive committee

and the steering committees are elected. The NACC Director and the NIA’s Program Director are ex-officio members of the Date Core steering committee. The Clinical Task Force is comprised of

five members who are elected by the members of the Clinical Core Steering Committee, six members who are appointed by the NIA and NACC, and three members who serve as liaison to other programs. Major Research Initiatives Involving the ADCs This collaborative infrastructure has provided the foundation for several major research programs that either directly or indirectly involve the ADC’s, the UDS, and the NACC.

8 NACC June 2013. https://www.alz.washington.edu/cgi -

bin/broker64?_service=naccnew9&_program=naccwww.pubrep1.sas&TYPEF=DISPLAYIDS

The Alzheimer’s Disease Genetics Consortium (ADGC) Genome Wide Association Study

(GWAS) was formed in 2003 to collaboratively use the collective resources of the AD research community to identify variability in genes that influence susceptibility to AD. The ADGC is a

partnership between the NACC, NCRAD, the ADCs, and the Children's Hospital of Philadelphia (CHOP). NACC identifies the subjects from each ADC who are eligible for the GWAS, tracks the

samples that have been sent to NCRAD for DNA banking, and reimburses the ADCs for their costs. NCRAD sends DNA for genotyping to CHOP, which sends the genotype data to the ADGC.

Finally, NACC provides the ADGC with the HIPAA research-limited phenotypic data, which it pairs with the genotype data.

Because AD is such a complex disease with significant phenotypic heterogeneity, it is a greater challenge for researchers to identify the genetic variants associated with AD necessitating a

combination of larger studies and novel approaches such as GWAS. (University of Pennsylvania School of Medicine, 2010).

The Alzheimer’s Disease Neuro-Imaging Initiative (ADNI) was launched the following year with approximately 50 research sites in the United States and Canada. It is a public-private

partnership with the NIA and the National Institute of Bioimaging and Bioengineering, 22 companies, and two Foundations including the Foundation for the National Institutes of Health

(FNIH), which administers non-government sources of funds such as the Alzheimer's Association. ADNI data (images, cerebrospinal fluid, and blood samples) are stored

anonymously in a central repository and all qualified researchers can access the electronically stored images and test results as soon as data are available.

The success of these collaborative multi-center research programs in the United States has

provided a model and infrastructure that has been scaled internationally. To that end, North American ADNI (NA-ADNI) served as a founding member of World Wide ADNI (WW-ADNI), a collaborative effort of scientists from around the world. It is now the umbrella organization for

neuroimaging initiatives being conducted by NA-ADNI, European ADNI (2005), Japan ADNI (2007), Australian ADNI (2006), Taiwan ADNI (2011), Korea ADNI, China ADNI (2011), and

Argentina ADNI (2012). Brazil ADNI (2014) is a future site. Since WW-ADNI was beyond the funding mandate of the NIA, the Alzheimer’s Association, a major charitable organization

dedicated to addressing AD, provided some of the initial funding for this global initiative.

In line with NA-ADNI, the goals of WW-ADNI are to better understand the progression of mild cognitive impairment and Alzheimer's disease, to standardize data collection methods so that data from all sites can be combined and provide a worldwide picture this disease. WW-ADNI is unique because the clinical, neuropsychological, imaging, and biological data are all available to the scientific community at no cost so that scientists around the globe can use this information

for their own research purposes.

The Dominantly Inherited Alzheimer Network (DIAN) is another international partnership with 10 research institutions in the United States, United Kingdom, and Australia researching a rare

form of AD caused by a gene mutation. DIAN was established in 2008 with a grant from the NIA.

It has received additional funding from foundations, charitable donations, and the German

Center for Neurodegenerative Diseases. The DIAN coordinating center is located at Washington University in St. Louis, which is also an ADC site.

Figure 3 provides a high level map of the AD research network and the tightly coupled

relationships between the ADCs, the centers and projects, and these collaborative research programs.

Figure 3. Mapping the Alzheimer’s Disease Research Network

Our focus, based on our initial scan and mapping of the AD research network/system, is the Uniform Data Set (UDS) project. It has played a central role in the emergence of this far-reaching collaborative research enterprise yet is sufficiently bounded so that we could meaningfully gather data on the key deliberations, both virtual and face-to-face, involved in creating it, implementing it, revising it, and ensuring timely and accurate data submissions from the ADC’s. It also involved a high degree of virtuality, multi-disciplinary activity, and varying levels of complexity and uncertainty in the work to be done and the processes required to complete the work.

These initial interviews also helped us confirm where to locate the UDS project on the six-stage

R&D continuum9 that we have been using in our research on virtual R&D projects. As shown in Figure 4, the six stages are demarcated by the degree of task uncertainty and complexity.

Figure 4: R&D Continuum10 with Location of Case Study Projects

Based on the initial scan and the results of the interviews, we concluded that creating, implementing, and

subsequently revising the UDS involved a degree of uncertainty characteristic of the initial development stages, or D1 and D2 on the R&D continuum. Once implemented, the activities involved in maintaining the UDS and ensuring timely submissions of accurate data were more routinized and akin to the start-up and scale-up stages (D3 and D4) on the R&D continuum.

We used the differences in the degree of complexity to sort the major deliberations that the interviewees identified as either the periodic deliberations involved in designing, implementing,

9 R&D has been characterized as an intrinsic learning system (Purser et al., 1992) with multiple stages. Each stage

is defined by the degree to which participants do or do not know the “what” (objective) or the “how” (method or means) of their knowledge development and synthesizing activities. These stages form a developmental

continuum that ranges from projects with high uncertainty in which participants don’t know what is the final or real objective in concrete terms and don’t know how to operationalize it – to projects with low uncertainty in which participants know what they need to achieve and also know how to achieve it operationally.

10 Carolyn Ordowich (personal communication, March 26, 2009) adapted the R&D continuum from a research

portfolio model originally developed at Bell Laboratories (Mashey, 2008; Revkin, 2008).

R

1

R

2

D

1

D

4

D

2

D

3

and revising the UDS or the ongoing deliberations needed to ensure the quality and timeliness

of the data submitted to the UDS.

Major Periodic Deliberations to Design, Implement, and Revise the UDS The NACC’s mission is to encourage and support more effective collaboration among ADCs

across the United States. The success of the UDS has been a critical contributor to achieving that mission, both symbolically and practically.

The UDS is the result of innumerable deliberations that have occurred since 1992, and probably many years before the planning of the UDS was formally initiated. As noted above, the process of designing and then launching the UDS took three years and required considerable effort on the part of numerous individuals from the ADCs, the NACC, and the NIA. This work was entrusted to the Clinical Task Force and various sub-committees comprised of subject matter experts on the main elements of the UDS.

The key deliberations, participants, values divergence and knowledge barriers, and the

coordinating mechanisms involved in designing, implementing, and revising/upgrading the UDS are summarized in Table 2. While the NACC and UDS have facilitated greater collaboration in the field of AD research, the task was fraught with challenges. Some of the ADC scientists disagree with the fundamental premise that standardized data collection and a uniform data base approaches is a more effective and productive way to conduct research on AD. It is not clear whether this constitutes a knowledge barrier due to lack of common frame of reference or due to divergent values . One person we interviewed observed that the UDS approach violated what he understood about the scientific method; namely that hypotheses and research questions should determine the measures and data to be collected, and not the other way around. Another interviewee described

the disagreement this way: The trade off is the perceived value of data from tens of thousands of people collected uniformly versus the value of 29 centers pursuing individual and local and maybe

smaller consortium expertise (Personal interview, 2011).

It is understandable that the scientists at the ADCs placed a high value on autonomy and the discretion to conduct research on AD as they deemed most appropriate. As some of the most eminent scientists in the field of AD researchers – demonstrated, in fact, by their ability to secure NIA funding for their ADCs -- they undoubtedly believe that they deserve this latitude. There is also the question of the cost of the UDS and the NACC, especially when research funds are limited. Creating an integrated system that allows collection, distribution, and shared use of the

data is also expensive, and not everyone believes that it is the best use of limited research

funds. One center director that we interviewed commented on the fact that more than one grant

was necessary to create and maintain the UDS.

Table 2. Major Periodic Deliberations to Design and Revise the UDS

Topics and Issues

Participants Values Divergence/ Knowledge Barriers

Coordinating Mechanisms

Fundamental premise of the UDS

ADC Directors and Scientists, NIA, NACC PI

Autonomy and research independence versus

standardized methods and measures Lack of shared frame of reference

Transcendent purpose, Threat of ADC not being

renewed Latitude to pursue personal research agenda as well as UDS

Design, build support,

and agree for the key elements of the UDS: Standardized clinical

evaluation protocol for assessing and

diagnosing AD and other dementias

Common clinical data

points Common data

collection forms

Relational database

Clinical Task Force,

discipline-specific sub-committees that solicit input and feedback from ADC researchers, NIA

PM, and NACC PI

Lack of a common frame

of reference (competing theories on the mechanisms of AD) Failure to use knowledge

from all disciplines

Multiple opportunities to

provide input and influence decisions Transcendent purpose, Credible and influential

network builders and leaders ADC steering committees

NACC NIA

Revise UDS to replace

proprietary instruments with non-proprietary instruments; anticipate and incorporate

scientific breakthroughs

Clinical Task Force plus

discipline-specific sub-committees that solicit input and feedback from Center staff

Sunk costs in

longitudinal data already collected versus rising l icensing costs and lost data due to inability to

extend licenses to non-ADCs Failure to use knowledge

from all disciplines

Multiple opportunities to

provide input and influence decisions; Transcendent purpose; Credible and influential

network builders, peer leaders; ADC steering committees; NACC; NIA

These deliberations occurred in informal discussions with peers at their own ADC and other ADCs and with individuals from the NACC and the NIA and in more formal settings such as the semi-annual directors meetings as well as numerous scientific meetings. The forums occurred both face-to-face (F2F) and virtually. There were three factors that allowed a workable resolution of the divergent values in this deliberation; first, the compelling mission that the AD

researchers share, the cure and eradication of this devastating disease; second, the latitude to pursue an ADC-specific research agenda along with the common agenda of the UDS; and third

the financial incentives of continued funding for the ADC

Countless deliberations were also required to design, build support, and agree on the key elements of the UDS which included a standardized clinical evaluation protocol for assessing

and diagnosing AD and other dementias, a common set of clinical data points, common data collection forms, and a relational database for the UDS.

This was arguably the most challenging set of deliberations. Reaching agreement on the value

of common standards in most decentralized organizations is conceptually a straightforward task. But in practice, people are only supportive of common standards and common

approaches if the chosen standards resemble their current practices. The scientists and ADCs had all invested considerable, time, money, and professional credibility to conduct the clinical

evaluations and gather the data in ways that best supported their research agenda and theories on the mechanisms of AD. Designing, building support, and reaching agreement on which

clinical evaluation protocol to use, which clinical data points to collect, how to configure the data collection forms, and how to construct the optimal relational data base to satisfy ADCs required an iterative process of building commitment to, or at least, compliance with the UDS.

The Clinical Task Force surveyed the ADCs to find out what protocols, data points, and

measures they were using at that time. They sought people’s input and feedback throughout the process. In this set of deliberations, the knowledge barriers were due to divergent values,

lack of common frame of reference, the failure to use the full range of knowledge resident in the ADC network. For example, some of the disciplines such as data managers did not believe

that their input was fully utilized in the final design of the UDS.

The UDS is evolving, and these changes are constrained by the choices made at the outset. The early decisions shaped the current form of the data set, and the well-established longitudinal

database made it difficult to make alterations to the UDS. But several of the instruments used in the UDS are proprietary and their use is governed by agreements that have become quite restrictive; in particular, the license fees for these instruments have increased dramatically. Here is how one interviewee described this challenge (Personal interview, 2011):

Almost all sites would use a Mini Mental State at some in their characterization of

patients. Well, it turns out that many of these instruments that we use, particularly the neuropsychological instruments, have been licensed by commercial firms. So we had to enter

into licensing agreements. Actually NACC, the National Alzheimer’s Coordinating Center, is the

one that developed these licensing agreements. And the companies varied in their willingness to do this. But some of them were really very restrictive. They’d had a bad experience with one

university or another, and they didn’t want them to be part of our licensing agreement. But in

the end of it all, it is that we do have blanket licensing agreements for all Alzheimer’s Disease

Centers to use the uniform data set in ADC participants. If you’re not in one of the ADCs, the licensing agreement doesn’t extend to you; you would have to go to the individual companies

and develop your own licensing agreement with each company that has an instrument in the

UDS. It’s a tremendous problem. It turns out that the licensing fees keep going up each year. It’s

absurd to us – I’m sorry, but this is a real problem for us. It’s absurd for us because many – I

think I mentioned the first centers began over 25 years ago, and we tried to adhere to using

the same instruments that they’ve been using for 25 years, so we didn’t ask to throw all of their longitudinal data sets out of the window. So we’re using in many instances, forms or

versions of these cognitive tests that have been subsequently revised. So the forms we use are

no longer being manufactured, except for us, and the companies – we’re the only users and

they keep raising the rates on us, even though they’re selling other versions, newer versions of

these forms. So it’s been a real problem. Many people have asked for access to the UDS; we’re

able to give them the parts, the components, that are not licensed, but most of the cognitive tests are licensed, so it’s a real, real problem. And talk about evolution, we have a current task

force, primarily made of neuropsychologists, that is trying to develop a new

neuropsychological battery for the UDS that would be comprised of tests that are unlicensed,

so that we could then disseminate the entire UDS without these restrictive licensing agreements.

One researcher remarked that standardization is also holding back change. The concern raised is that changing the items or definitions of elements in the items can damage long-term

studies by creating uncertainty about the meaning of the data. Another concern raised was the cost of change in terms of the reorganization of the data set, the forms used to gather the data,

and staff needed to make the changes.

The Clinical Task Force and subject area sub-committees conducted a lengthy set of deliberations to develop and pilot a non-proprietary battery of instruments. Most of these deliberations were conducted virtually using email, teleconferences, and occasionally videoconferences. They also met in person at the Directors meetings. They experienced a similar knowledge barrier revising the UDS as they did developing the original UDS; namely, the failure to use the knowledge and expertise of all of the disciplines involved in the ADCs. For example, one interviewee commented that they did not believe that any data managers from

the ADCs were included in the sub-committees.

But the need to anticipate scientific developments and modify the UDS accordingly is an ongoing challenge. In a field as vitally important as AD research, there is unrelenting pressure

to stay on the forefront of scientific discoveries and technological advances. As one

interviewee stated (Personal interview, 2011),

. . . it’s a hard row to hoe because a lot of that literature is in computing journals and engineering journals and linguistic journals, and it’s hard to get the mainstream

biomedical community to sort of turn the battleship around. It’s an interesting time we live in. But I think one of the amazing transformations that’s going to happen is

that hopefully with the emphasis on the electronic media we will begin to realize that there’s a tremendous power in aggregating data electronically, but actually the

electronic capture of data doesn’t just have to occur in a clinic. It can actually occur anywhere at any time.

These anticipatory deliberations can conceivably involve all member of the AD Research

network; especially the formal and informal network leaders. These deliberations have occurred and will continue to occur both within the formal structure and the informal influence

network.

Major Ongoing Deliberations to Maintain the UDS

The major ongoing deliberations required to maintain the UDS and ensure data quality and timely submissions to the UDS are summarized in Table 3.

Implementing the original UDS as well as revised versions of the UDS involved deliberations to

educate ADC personnel regarding conducting standardized clinical evaluations, collecting the common clinical data points, using the common data collection forms, and inputting the data into the UDS’s relational database. The Clinical Task Force and the NACC developed detailed

user manuals for these activities which they made available to the ADCs on their website. While most of these deliberations were conducted virtually, there were discussions regarding

implementation and revisions with the relevant Core Directors at the semi-annual Director’s Meetings. The NACC also created an online bulletin board that the ADCs could use to submit

questions and get answers on data collection issues. In the early days of the UDS implementation, there was considerable traffic on the bulletin board about how to deal with

various scenarios. If necessary, the NACC forwarded the question to the Clinical Task Force for a decision. Because they have been collecting UDS data for several years, the bulletin board is relatively

quiet; that is, until the NACC introduces minor modifications to the UDS. The number of questions posted on the online bulletin board is contingent on the complexity of the modifications. We were informed in our interviews that the decisions to make minor modifications to the UDS are made centrally by the NACC and the Clinical Task Force and then communicated by group emails to the ADCs. However, the group emails may not be sent to all parties impacted by the decision. For example, if it is a clinical issue, data core staff may not be

copied on the changes. This may lead to downstream problems when the data manager attempts to enter the modified data formats. Attempting to implement and maintain a uniform data set across 29 geographically dispersed, stand-alone ADCs is challenging. More specifically, while the Clinical Task Force and the NACC standardized the outputs with the common data forms, they have not standardized the local processes. The lack of standardization in how different types of expertise are integrated and deployed across the ADCs was an issue that surfaced during our interviews.

Each ADC has its own unique clinical processes for assessing and diagnosing AD and collecting and managing the data. The data is collected and analyzed at the ADCs by multidisciplinary

groups of people: clinicians, neurologists, statisticians, data managers, neuropsychologists, nurses and research staff. The Centers have differing professional categories and

responsibilities for their staff. For example, some centers include the statisticians and data managers in the team and others view them as support staff only leading to varying degrees of

involvement in deciding how best to code the items for the database. One statistician described

their role this way: “I’m the outsider looking in, I guess.” By comparison, the data manager at another ADC meets with the clinical staff twice a week to review patient charts from which the data to be submitted to the UDS is generated.

Table 3. Major Ongoing Deliberations to Maintain the UDS

Topics and Issues

Participants

Forums and Degree

of Virtuality

Values Divergence/

Knowledge Barriers

Coordinating

Mechanisms

Implement UDS and major revisions to the UDS

NACC staff, Clinical Task Force, ADC clinical core and

data core staff

Online users manuals and bulletin board for

questions, emails, and F2F updates at Directors meetings

Lack of knowledge due to new or updated standards

Well-defined procedures, online manuals, F2F

updates and online

Implement minor modifications to the

UDS

NACC staff, Clinical Task Force, ADC

clinical core and data core staff

Emails and online bulletin board

Failure to share knowledge with all

relevant parties

NACC specifies requirements

Consistency of diagnosis, accuracy and completeness of

data forms

ADC clinical core staff and data core staff

Email, F2F, hard copies of the data forms

Lack of common frame of reference due to diversity of

disciplines and ADC-specific processes

NACC provides specific requirements

Quality and timeliness of data submitted to ADC by

ADCs

ADC staff (data mgmt. staff, clinical staff, Core and

Center Directors), NACC staff and PI, and NIA PM

Primarily virtual (EDI, monthly reports, emails, and

telephone if an issue needs to be escalated

Familiarity of staff with data forms; Failure to share

data/knowledge in timely fashion due to differing priorities

NACC uses error tracking procedures;

ADC’s commitment to data integrity

While decisions about team inclusion are made in each center, the NACC and UDS might benefit from some analysis of the tasks that are vital to ensuring consistency of the data set. In this situation, statisticians and data managers make decisions about the definitions that are core to

interpretation of the data.

Also, the degree of co-location differs from center to center. At some ADCs, staff members are co-located while at other ADCs they are in separate departments that are housed in different

buildings and located on other areas of the campus. The degree of geographic dispersion affected both the amount of interaction and the modes of interaction.

These factors impact the reliability of the UDS. To mitigate these risks to the UDS, both the

ADCs and the NACC have evolved a process that they follow each month to ensure data quality. The data core staff at the ADCs conduct quality checks on their data before they submit it via electronic data interchange to the NACC each month. Once they receive the data from the ADC, NACC’s data managers and programmers conduct a standardized set of quality checks and then distribute a report to the Center Directors, Clinical Core Directors, and the Data Core Director. These reports will typically be shared with other Center staff such as the data core

manager, clinical core leader, scheduling assistants, and so on. If the NACC discovers any data

discrepancies, they are listed in these reports to the ADCs. ADC staff will compare the data

they submitted with what the NACC reports and then email questions to the programmers and

data managers at NACC.

The NACC reports also include questions regarding what they consider to be inconsistencies in the data. One example involved how a Center’s clinicians classified subjects as depressed.

When asked if a subject exhibited depressive symptoms, the clinicians said yes. When asked in a later question if the patient was depressed, the clinicians indicated that s/he was not. The

data managers asked the clinicians for clarification and were told that the subjects had signs of depression, but they were not comfortable diagnosing the subject as clinically depressed because they did not know if the person had a history of clinically diagnosed depression. Here is how one of the interviewees described another virtual deliberation that their ADC had with the NACC (Personal interview, 2011): “We have been back and forth with them (NACC) on how they clarify follow-up rates and how they count who is being followed in what time window. The time window is where we really, really struggled with how they ended up with

different rates reported from how we thought they should be. But ultimately their decision is

the one we have to follow.” The issue of follow-up rates was particularly relevant because the ADCs must report a specific number of patients enrolled in their Center’s data set each month. Failure to do so signals potential problems and is an important consideration in future funding. There were probably several reasons that an ADC might be tardy in subject follow-up or timeliness of their data submission: the disagreement on the calculation of follow up rates, the amount of effort required to be fully up-to-date, and conflicting priorities for how they wanted to spend their time. The original deliberations occurred initially with email requests from the NACC to the ADCs. If the ADCs failed to comply or were slow to comply, the deliberations escalated to telephone calls from the NACC’s Director to the Center’s Director. If this did not produce the desired results, it would be followed by telephone calls from the NIA’s Program Manager.

Given the threat of losing center funding, subject follow-up and timeliness of data submissions are less problematic now than when the UDS was originally implemented.

DISCUSSION AND CONCLUSIONS This case provides illustrative insights into the complexity of deliberations involved in virtual R&D

projects. It is generally acknowledged that the UDS, and by extension the deliberations involved in

creating it, implementing it, maintaining it and revising it, have been successful. An August 2010, New York Times article reveals that while scientists may have had initial concerns about such a collaborative effort, they have started to recognize the benefits. One researcher quoted in the

article noted “it’s not science the way most of us have practiced it in our careers. But we all realized that we would never get biomarkers unless all of us parked our egos and intellectual property noses outside the door and agreed that all of our data would be public

immediately.” This attitude has prevailed and the Alzheimer’s project has become a model for other disease states such as Parkinson’s disease.

Similarly, the following observation from our interviews attests to the success of the UDS

(Personal interviews, 2011): “I think the metric of success is the extent to which the data we’re collecting is getting used and advancing scientific productivity. We look at how many different

proposals have come forward to use the data and how many publications have come out that have used the data inside of the grant. I think that’s the main reason we’re doing this. It is not

just collecting data for collecting data sake, but actually to answer the questions. The first couple of years were just about getting the data collected in a format that would be useable,

and now we’re beginning to see the fruits of that labor, if you will.” Coordinating Mechanisms The effectiveness of the UDS deliberations is due in part to the coordinating mechanisms that provided the context for the key stakeholders to make the important trade-offs on a reasoned and ongoing basis. The theory of organizational information processing and mutual meaning making (Galbraith, 1974; Boland and Tenkasi, 1995; Malhotra & Majchrzak, 2004; Weick, 1995, 1979; Daft & Lengel, 1986) suggests that the structural mechanisms for coordination must

provide the means to address the level of task uncertainty (what) and the complexity of

achieving the task (how) in order to effectively co-create knowledge (Nonaka, 1994; Boland and Tenkasi, 1995). Sabherwal (2003) condensed several typologies of coordinating mechanisms into a continuum of four main types: informal mutual adjustment, formal mutual adjustment, plans, and standards. Figure 5 superimposes these four coordinating mechanisms onto the R&D continuum in light of the levels of task complexity and uncertainty. The key stakeholders in the AD research network generally knew what to do to create, implement, and subsequently revise the UDS. How to do it, especially how to build support for the UDS given the divergent values and priorities of the 29 geographically dispersed ADCs, was more challenging and complicated (stages D1 and D2 on the R&D continuum). The coordinating mechanisms for these deliberations relied heavily on formal mutual adjustment and to a lesser, but no less important, degree on informal mutual agreement. The key structural elements of

the formal mutual adjustment included the national coordinating structure (NACC), the ADC steering committees, the semi-annual Directors meetings, the Clinical Task Force and its sub-

committees, the AD center model, and the NIA.

Arguably the most powerful coordinating mechanism has been the compelling mission that is shared by the entire AD research community: eradicating Alzheimer’s disease. This underscores both the informal and informal coordinating mechanisms and is the basis for coordination of efforts requiring shared goals and shared plans.

Figure 5. Coordinating Mechanisms Across the R&D Continuum

The NACC has played a particularly important structural role in the formal mutual adjustment involved in creating the UDS. The NACC has provided the requisite infrastructure for effective and efficient deliberations within this virtual inter-organizational domain. In so doing, it serves

the key functions of a referent organization, which Trist (1983) identified as the coordination of relationships and activities, appreciation of emergent trends and issues, and infrastructure support including providing resources, sharing information, and conducting special projects. As mentioned above, the NACC organizes the semi-annual, F2F meetings of the ADC core directors

and ADC steering committees. It provides funding to the ADCs to offset the cost of submitting their data to the UDS, it funds collaborative research projects, and it assists scientists seeking to use the UDS in their research. The distinctions between informal and formal mutual adjustment are not so much binary, as incremental or additive, in that the necessary conditions for effective informal mutual adjustment such as trust and respect, also provide a positive foundation for more formal mechanisms of mutual adjustment. The foundation for informal mutual adjustment among the

ADCs and the NACC was a dense network of collegial relationships among participants in the AD research network that had developed over several years of interacting with each other at

conferences and symposiums, in professional organizations on editorial boards, and on review panels for the NIA and other funding agencies. In addition, many of the ADC core directors and

center directors serve on the Advisory Boards of two to three other ADCs, much like the interlocking directorates in the private sector.

This level of F2F interaction has certainly served to reduce some of the complexity in the working relationships that might otherwise have adversely affected the efficacy of the

deliberations in this virtual R&D project; that is, the members of the teams tasked with creating

the UDS were familiar with each other and had already developed socially derived norms of

interaction.

Further, the participants who were elected or selected to serve on the formal coordinating mechanisms – the Clinical Task Force the working sub-committees, and the ADC Steering

Committees are generally highly regarded members of the AD research network who possessed considerable source credibility based on their acknowledged expertise and perceived

trustworthiness (Gilbert et al, 1998). An interviewee described the qualities of these individuals as follows:

The task forces were comprised of people that were well recognized as being leaders and reasonable people. I think that those are the two characteristics of the people on the task forces; they’re not only leaders, they’re politically astute. They’re open. They’re “just reasonable” is the best way to describe them. They’re not difficult to deal with. Because they commanded confidence from the various other members that were involved in

contributing data, they’ve been successful in driving each other forward.

Another informal and formal mechanism for mutual adjustment was the presence of several very effective network leaders and boundary spanners from the NIA, the NACC and the ADCs. These network leaders or boundary spanners serve a ‘reticulist’ (Friend et al., 1974) role that has been critical to the creation and implementation of the UDS from vision to inception. Alter and Hage (1993) describe reticulists as “individuals who engage in networking tasks and employ methods of co-ordination and task integration across organizational boundaries” (p. 46). Similarly, Webb (1991) describes reticulists as “individuals who are especially sensitive to and skilled in bridging interests, professions and organizations” (p. 231). Both are apt descriptions of the key network leaders in the AD research network who have provided an effective balance of diplomacy, negotiation, persuasion (Rhodes, 1999), and when necessary confrontation to ensure the success of this project and the AD research network.

Once the UDS had been established, the NACC and the ADCs generally knew what to do and how

to do it both conceptually and operationally (stages D3 and D4 on the R&D continuum, Figure 4). The tasks and deliberations required to maintain the UDS have become more routinized and are

less uncertain and less complex. These deliberations are coordinated with plans and standards including specific requirements, delivery schedules for data submissions, and financial incentives. In fact, coordination by standards is the fundamental premise of the UDS with its standardized clinical evaluation protocol for assessing and diagnosing AD and other dementias, common data points, standardized data collection forms with detailed coding manuals, and error tracking

procedures. And with regards to the UDS, the relationship between the NACC and the ADCs, while still collegial, has generally become more hierarchical and prescriptive. As the interviewee stated

above: “… ultimately their decision is the one we have to follow.”

The findings from this case study on coordinating mechanisms support Sabherwal’s (2003) assertion that more informal, communications-oriented coordinating mechanisms are suitable

when uncertainty is greater, and that the more impersonal and formal, control-oriented

mechanisms are a better fit when the level of task uncertainty is lower.

Virtuality and Communication

The nature of communication, whether F2F or technologically mediated and the choice of forums for the deliberations, were also impacted by the levels of task complexity and uncertainty. The

deliberations to create, implement and revise the UDS relied on high levels of input and participation directly for the members of the Task Force and the sub-committees and for others via

surveys, bulletin boards, and direct feedback to the Clinical Task Force and its sub-committees. These discretionary coalitions used emails, teleconferences, and occasionally videoconferences for their deliberations. However, when they had a particularly difficult decision to make they usually made these in F2F meetings. As a member of one of the working groups stated: “One of the things I demanded in my job on the toolbox has been face-to-face meetings of my team at regular intervals. We’ve been doing everything by email and phone conferences, but there is nothing like a face-to-face meeting.” It is also interesting to note that even though there were several information and communication technology (ICT)-mediated vehicles for providing input, the ADC

Directors often used the public forums in their semi-annual meetings to raise issues or share their

concerns.

The communication involved in the ongoing deliberations to maintain the UDS are predominantly

technologically mediated: that is, the data is submitted electronically to the relational data base; questions for clarification are posed and answered on an online bulletin board; and discussions

between the ADCs and the NACC regarding submitted data are conducted almost exclusively by email and only in exceptional cases by telephone.

Even the day-to-day operations of the ADCs rely heavily on information and communication

technology. Here is how an interviewee described communications within their ADC (Personal interview, 2011): Communications are partly face-to-face. They’re partly by email. They’re partly by a system that my data management group created, which is online. It’s partly by electronic

reports that are also up on the Web. They’re generated each evening. And (in our center) they’re partly through an online data management request system that we built, where you couple

communications with data management.

In sum, the routine deliberations characteristic of the D3 and D4 stages in the R&D continuum were mostly virtual and technologically mediated. In comparison, the deliberation topics that

were more uncertain and complex (at the D1 and D2 stages in the R&D continuum), especially those that involved important and sensitive trade-off decisions, tended to be addressed in F2F

interactions.

Culture of Collaboration The NIA has been very explicit in its intention to foster an environment of collaboration across

multiple disciplines and the network of ADCs. Tony Phelps, NIA Program Director, described the

NIA’s vision for an ethos of collaboration on the 20th anniversary of the ADCs as follows:

When the ADC program was created, NIA’s leaders hoped that an environment of

cooperation would stimulate AD researchers to seek new pathways to scientific discovery and to share their findings. The program has done just that. In the collaborative

atmosphere of the centers, specialists in biomedical, behavioral, pathological, and clinical science are studying the causes, and possible prevention of AD, and developing new lines

of multidisciplinary research. As scientists at the centers uncover the complexities of dementia, they spark a certain friendly competition with other research scientists

throughout the world. Their brilliance and enthusiasm creates an excellent training ground for up and coming investigators, and inspires others to devote their energies to AD research (National Institute of Aging website, 2005)

The following comment from our interviews provide anecdotal evidence that the NIA’s vision of cooperation and ‘friendly competition’ is being realized (Personal interview, 2011):

I think we’re beginning to see young investigators and more established investigators

getting grants to do secondary data analysis, using the information, planning GWAS

studies, hooking up to other investigators within the group who have large data sets that they’ve contributed, looking to see if they have other data that’s not necessarily part of the UDS, but would help to augment analyses that they’re doing. I think we’re seeing a lot of that now, and that will definitely be the metric for success going forward. It’s becoming more and more important.

This researcher’s comment also captures the balance of competition and cooperation that now characterizes the ADC network:

I would say cooperation is really the key behind what we’re doing right now in terms of the UDS. And I think because we’re all part of the same project, it doesn’t make any sense to be competing with one another because we all have the same common goal

whether we like it or not. We’ve got to collect the data for this protocol. Now, we may compete in other ways in terms of other aspects of scientific research, but for this

protocol and for this project, we’re all on the same page. So we all have to work together for a common good. I think it’s that spirit that we’re all one family for this

project, so we have got to figure out how we’re going to work together. Building and maintaining the UDS and this collaborative network of researchers has been a social task as well as one of technical or professional alignment. Admittedly there have been pressures that would normally work against cooperative effort such as competition for funding

and recognition such as the Nobel Prize for Medicine, ego, professional jealousy and such. It appears that these effects have been mitigated in large part by appealing to the compelling

mission and genuinely shared goal of eradicating AD, as well incentives such as continued funding for one’s center.

The UDS has played a significant role in fostering and extending the culture of collaboration in

the AD research network. It has also played a role in other major collaborative research

initiatives both in the United States and now globally, including the Dominantly Inherited

Alzheimer Network, the Alzheimer’s Disease Genetics Consortium Genome Wide Association Study, the Alzheimer’s Disease Neuro-Imaging Initiative (ADNI), and now, World Wide ADNI.

The UDS is a brilliant example of the benefits to be gained from virtual collaboration across

research sites and disciplines. Together, all of these contributors have created a powerful instrument to help accelerate the identification of new treatments, preventions and ultimately,

a cure for this insidious disease.

References

Alzheimer's Disease International. World Alzheimer Report 2009. www.alz.co.uk/research/world-report

Asaro, F. 2011. Universal Co-opetition: Nature's Fusion of Competition and Cooperation. Betty Youngs Books.

Brandenburger, A. and Nalebuff, B., 1997. Co-Opetition: A Revolution Mindset That Combines Competition and Cooperation, New York: Doubleday Currency

Dominantly Inherited Alzheimer Network, http://www.dian-info.org/

Evans, D.A., Funkenstein, H.H., Albert, M.S., Scherr, P.A., Cook, N.R., Chown, M.J., Hebert, L.E., Hennekens, C.H., and Taylor, J.O. 1989. Prevalence of Alzheimer's disease in a community

population of older persons. Higher than previously reported. JAMA. Nov 10; 262(18): 2551–2556.

Friend, J., Power, J., and Yewlett, C. 1974. Public Planning: The Intercorporate Dimension,

London, Tavistock.

Gilbert, D.T., Fiske, S.T., Lindzey, Gardner, eds. (1998). The Handbook of Social Psychology. Oxford University Press.

Kolata, G., 2010, Sharing of Data Leads to Progress on Alzheimer’s, August 12, 2010, New York

Times.

Mashey, J. (2008, December 3). What can we learn from Bell Labs about managing energy R&D? Retrieved from http://bravenewclimate.com/2008/12/02/hansen-to-obama-pt-iv-where-to-

from-here/#comment-3023.

National Institute of Aging, 2013, Alzheimer’s Disease Research Centers. Retrieved from

http://www.nia.nih.gov/alzheimers/alzheimers-disease-research-centers

National Institute of Aging, 2013. Retrieved from http://www.nia.nih.gov/alzheimers/about-adear-center

Purser, R.E, Pasmore, W.A., Tenkasi, R.V., 1992. The Influence of Deliberations on learning in

new product development teams, Journal of Engineering and Technology Management, 9, pp. 1-28.

Revkin, A. (2007, December 12), ‘R2-D2’ and other lessons from Bell Labs. New York Times.

Retrieved from http://dotearth.blogs.nytimes.com/2008/12/12/r2-d2-and-other-lessons-from-bell-labs/.

2013 www.alz.washington.edu/WEB/NACCsum.pdf

Alzheimer’s Association re WW Adni http://www.alz.org/research/funding/partnerships/ww-

adni_overview.asp

2010 University of Pennsylvania School of Medicine http://alois.med.upenn.edu/adgc/

Williams, P. 2010. Special agents: The nature and role of boundary spanners. Paper presented at the ESRC Research Seminar Series: Collaborative Futures: New Insights from Intra and Inter-Sectoral Collaborations, University of Birmingham.

Appendix A - Methodology

We incorporated the diagnostic steps of open sociotechnical systems design to gather data and analyze the impact of virtuality on the AD research network: conduct an initial scan of the system, map the system, analyze the technical subsystem, and analyze the social subsystem (Pava, 1983). We did this in two stages. First, we conducted an initial scan of the system, the AD research network, and used this preliminary information to map the system and choose a

specific project to investigate in greater depth. We then conducted a series of interviews to analyze the technical and social subsystems.

Conducting an Initial Scan and Mapping the System

The purpose of an initial scan is to discern the mission or goals of the system and the governance processes and coordination mechanisms that enable collaboration in pursuit of the mission. The mission and governance system provide the impetus for a self-regulating system of players who define and iteratively evolve the technical subsystem in terms of the key

deliberations or issues they need to address in order to achieve the mission. We conducted the initial scan of the AD research system/network by reviewing available documentation on the major research initiatives and key stakeholders, by attending and

observing the Fall 2010 ADC Directors meeting in San Francisco, and by interviewing the Principal Investigator of the National Alzheimer’s Coordinating Center (NACC) and the NIA’s

Program Director for the Alzheimer’s Disease Centers.

This afforded us a deeper understanding of the mission of the AD research network and the emergence of the ADCs, the NIA-funded centers, and other major research programs, initiatives and projects such as the UDS, ADNI, GWAS, NCRAD, and DIAN. Analyzing the Technical Subsystem

Pava (1983) described deliberations as “equivocality reducing events”, i.e. choice points that are critical to work systems involving knowledge generation and knowledge utilization. From

this general description, Purser (1990) defined deliberations in product development as “social interactions in which knowledge is exchanged to define or solve a problem, make a decision, or

implement a solution”. However, deliberations are not simply the equivalent of decisions or meetings; rather, they are sense-making exchanges (Weick, 1995), communications and

reflections in which people engage to reduce the equivocality of a problematic iss ue.

A deliberation is identified by the existence of an equivocal topic, which is explored in different types of forums, involving a particular group of participants who contribute important

information or take-away important information. Deliberation analysis assesses the values and perspectives of participants within forums, and “the interpretative dynamics among interdependent parties who must forge a discretionary coalition” (Pava, 1983) to make

intelligent trade-offs from their respective values, priorities, and cognitive orientations (Tenkasi, 1994; 2000).

Deliberations connect people together to meaningful issues much like linear work processes

connect people together to a common outcome. For people who work virtually and may never see one another as they work together, this connection to an issue may be vital to their

performance.

Deliberations in knowledge work such as R&D can be viewed in terms of intellectual bandwidth (Nunamaker et al. 2001, 2002, Qureshi et al. 2002) and the ability to mobilize intellectual assets

in deliberations to create value. This model provides a framework for measuring the extent to which an organization can create value from its intellectual assets by looking at two key elements in deliberations. The first is the process of understanding the data and available information and translating it into knowledge. The second addresses the interdependence of efforts and whether it is primarily an individual work mode, a collected work mode and the sum of individual work, a coordinated work mode in which there is sequential interdependence or a concerted work mode in which all work in concert to produce joint deliverables.

Analyzing the Social Subsystem The social system is defined in terms of discretionary coalitions that are needed to conduct the deliberations effectively. These coalitions make the important trade-offs in creative work that are made necessary by the presence of useful but inherently divergent values and perspectives.

For example, in traditional research environments, scientists typically compete against one another for limited grant money and to publish articles in top journals, neither of which enable

the effective functioning of coalitions in a virtual project. The social system design does not try to eliminate differences, but to create a mutual understanding and a common orientation such

that trade-offs can be settled on an intelligent and ongoing basis. Coalitions are to nonlinear work what work groups or teams are to more routine work. Roles and responsibilities can be

defined for the parties involved in the coalitions as well as other changes in the coordinating

mechanisms in a way that supports and rewards the sort of integrative perspective necessary to successful coalition functioning.

In classic sociotechnical systems analysis, the focus is on addressing variances in work processes and performance. However, in the knowledge work environment of R&D projects (co-located

and virtual), variances manifest as knowledge barriers. Purser et al. (1992) identified four main

categories of “barriers” obstructing and delaying collaborative knowledge development: The first was the failure to share knowledge due to lack of cooperation, missing parties, or

unrealistic timeframes; the second was the lack of a common frame of reference including cognitive frame of reference barriers associated with differences in language, values, and

functional expertise; third was the lack of knowledge about the work, procedures and processes, or the capabilities of virtual participants that can slow or derail progress; and the

fourth was failure to utilize knowledge.

In order to develop an interview frame, we performed an extensive literature review on deliberations in virtual R&D organizations. We incorporated interview questions that would be used in a sociotechnical scan of the technical subsystem in non-routine knowledge work. This included listing the major deliberations, identifying the different forums in which these

deliberations were conducted (in person, videoconferences, teleconferences, email, etc.),

determining the structures of the forums (structured, semi-structured, and unstructured), identifying the participants in these key deliberations, and surfacing the variances or knowledge

barriers in the deliberations.

To analyze the social subsystem within which the major deliberations occurred, we developed questions that addressed issues such as the role of discretionary coalitions, key influencers or

participants, means of building support for decisions, and how they managed conflicting objectives or perspectives. Since the structure of the UDS had already been established and many of the early implementation challenges had been addressed, we conducted retrospective interviews and secondary data searches to understand the development process, the scope of involvement of various personnel, and the kinds of decisions and actions taken to reach consensus on the UDS.V1. However, at the time we conducted our interviews, the NACC and the Clinical Task

Force were in the process of revamping the batteries of instruments and measures for the

UDS.V3 and we were able to gather contemporaneous insights on how they were conducting this task in a virtual environment. We also interviewed ADC researchers about current use, decisions, and applications of the UDS to gain a perspective on how deliberations related to it have changed over time. We followed a protocol established by the VOSS research team, but customized the questions for the ADC interviews in light of our discussions with the NACC PI and NIA’s Program Director, the insights gathered at the ADC Directors semi-annual meeting, and our preliminary mapping of the ADRN system. We also asked for their input on selecting a representative sample of individuals who have been involved in the key deliberations related to the UDS. A total of 12 telephone interviews were conducted during the winter and spring of 2010-2011 with a cross-section of ADC center directors, principal investigators, clinical core directors, neurologists,

psychiatrists, biostatisticians and other members of the research teams.

The interviews yielded considerable data about the technical and social subsystems. The interviews were transcribed and members of the research team analyzed the content to

identify common themes regarding the major deliberations, the various forums in which they occurred, the role and relative efficacy of virtual or technology-mediated forums and face-to-face meetings, the sources of variances or knowledge barriers, and the coordinating mechanisms used to address them.