+ All Categories
Home > Documents > Peer impressions in open source organizations: A survey

Peer impressions in open source organizations: A survey

Date post: 30-Dec-2016
Category:
Upload: lorin
View: 215 times
Download: 1 times
Share this document with a friend
12
Please cite this article in press as: Bosu, A., et al., Peer impressions in open source organizations: A survey. J. Syst. Software (2014), http://dx.doi.org/10.1016/j.jss.2014.03.061 ARTICLE IN PRESS G Model JSS-9314; No. of Pages 12 The Journal of Systems and Software xxx (2014) xxx–xxx Contents lists available at ScienceDirect The Journal of Systems and Software j ourna l ho mepage: www.elsevier.com/locate/jss Peer impressions in open source organizations: A survey Amiangshu Bosu a , Jeffrey Carver a,, Rosanna Guadagno b , Blake Bassett e , Debra McCallum c , Lorin Hochstein d a Department of Computer Science, University of Alabama, Tuscaloosa, AL, USA b National Science Foundation, Arlington, VA, USA c Institute for Social Science Research, University of Alabama, Tuscaloosa, AL, USA d SendGrid Labs, Providence, RI, USA e Department of Computer Science, University of Illinois Urbana-Champaign, Champaign, IL, USA a r t i c l e i n f o Article history: Received 1 December 2012 Received in revised form 8 January 2014 Accepted 19 March 2014 Available online xxx Keywords: Open source software Impression formation Virtual teams a b s t r a c t In virtual organizations, such as Open Source Software (OSS) communities, we expect that the impres- sions members have about each other play an important role in fostering effective collaboration. However, there is little empirical evidence about how peer impressions form and change in virtual organizations. This paper reports the results from a survey designed to understand the peer impression formation process among OSS participants in terms of perceived expertise, trustworthiness, productivity, experi- ences collaborating, and other factors that make collaboration easy or difficult. While the majority of survey respondents reported positive experiences, a non-trivial fraction had negative experiences. In particular, volunteer participants were more likely to report negative experiences than participants who were paid. The results showed that factors related to a person’s project contribution (e.g., quality and understandability of committed codes, important design related decisions, and critical fixes made) were more important than factors related to work style or personal traits. Although OSS participants are very task focused, the respondents believed that meeting their peers in person is beneficial for forming peer impressions. Having an appropriate impression of one’s OSS peers is crucial, but the impression formation process is complicated and different from the process in traditional organizations. © 2014 Elsevier Inc. All rights reserved. 1. Introduction Many expert developers devote a significant amount of effort to Open Source Software (OSS) projects. Because many of those participants are not directly compensated, their participation must be motivated by other factors. Previous empirical research found that OSS participants are motivated by the prospects of enhanc- ing their reputation and by being identified with a particular OSS community. According to Raymond (1999): “The ‘utility function’ Linux hackers are maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers.” Reputation among one’s peers is the only available mea- sure of competitive success (Raymond, 1998) and the main source of power (Evans and Wolf, 2005) in OSS communities. Furthermore, the participants’ desire to maintain a good reputation among their peers is a major motivation for voluntarily devoting effort to an OSS Corresponding author. Tel.: +1 205 348-9829. E-mail addresses: [email protected] (A. Bosu), [email protected] (J. Carver), [email protected] (R. Guadagno), [email protected] (B. Bassett), [email protected] (D. McCallum), [email protected] (L. Hochstein). project (Gutwin et al., 2004). The level of dedication a participant has for the OSS project is strongly related to peer recognition (Xu and Jones, 2010). Therefore, gaining and maintaining reputation is a key factor in keeping an OSS project on track (Markus et al., 2000). When a participant is well recognized within the OSS community, his or her peers regard the participant’s project related opinions more carefully (Gacek and Arief, 2004). Because gaining peer recognition is a major motivation for OSS participants and it influences OSS projects greatly, it is important to understand the peer recognition process within OSS communities. We define peer recognition as: the acknowledgement of a person’s merits or status by his or her peers. Before a person can acknowledge the merits or status of a peer s/he must be aware of those mer- its or status. Therefore, it is important to understand how peers form opinions of each other in OSS communities. There is a large body of research on peer recognition in the Psychology literature, where researchers use the term “impression formation” to describe the same concept. According to Kenny (1994), peer impression is “the judgements that a person, called the perceiver, makes about another person, called the target, where the target is a real person”. The formation of interpersonal impression primarily depends upon how well the perceiver is acquainted with the target and upon the http://dx.doi.org/10.1016/j.jss.2014.03.061 0164-1212/© 2014 Elsevier Inc. All rights reserved.
Transcript
Page 1: Peer impressions in open source organizations: A survey

J

P

ADa

b

c

d

e

a

ARRAA

KOIV

1

tpbticLihsotp

(d

h0

ARTICLE IN PRESSG ModelSS-9314; No. of Pages 12

The Journal of Systems and Software xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

The Journal of Systems and Software

j ourna l ho mepage: www.elsev ier .com/ locate / j ss

eer impressions in open source organizations: A survey

miangshu Bosua, Jeffrey Carvera,∗, Rosanna Guadagnob, Blake Bassette,ebra McCallumc, Lorin Hochsteind

Department of Computer Science, University of Alabama, Tuscaloosa, AL, USANational Science Foundation, Arlington, VA, USAInstitute for Social Science Research, University of Alabama, Tuscaloosa, AL, USASendGrid Labs, Providence, RI, USADepartment of Computer Science, University of Illinois Urbana-Champaign, Champaign, IL, USA

r t i c l e i n f o

rticle history:eceived 1 December 2012eceived in revised form 8 January 2014ccepted 19 March 2014vailable online xxx

eywords:pen source software

mpression formationirtual teams

a b s t r a c t

In virtual organizations, such as Open Source Software (OSS) communities, we expect that the impres-sions members have about each other play an important role in fostering effective collaboration. However,there is little empirical evidence about how peer impressions form and change in virtual organizations.This paper reports the results from a survey designed to understand the peer impression formationprocess among OSS participants in terms of perceived expertise, trustworthiness, productivity, experi-ences collaborating, and other factors that make collaboration easy or difficult. While the majority ofsurvey respondents reported positive experiences, a non-trivial fraction had negative experiences. Inparticular, volunteer participants were more likely to report negative experiences than participants whowere paid. The results showed that factors related to a person’s project contribution (e.g., quality and

understandability of committed codes, important design related decisions, and critical fixes made) weremore important than factors related to work style or personal traits. Although OSS participants are verytask focused, the respondents believed that meeting their peers in person is beneficial for forming peerimpressions. Having an appropriate impression of one’s OSS peers is crucial, but the impression formationprocess is complicated and different from the process in traditional organizations.

. Introduction

Many expert developers devote a significant amount of efforto Open Source Software (OSS) projects. Because many of thosearticipants are not directly compensated, their participation muste motivated by other factors. Previous empirical research foundhat OSS participants are motivated by the prospects of enhanc-ng their reputation and by being identified with a particular OSSommunity. According to Raymond (1999): “The ‘utility function’inux hackers are maximizing is not classically economic, but is thentangible of their own ego satisfaction and reputation among otherackers.” Reputation among one’s peers is the only available mea-ure of competitive success (Raymond, 1998) and the main source

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

f power (Evans and Wolf, 2005) in OSS communities. Furthermore,he participants’ desire to maintain a good reputation among theireers is a major motivation for voluntarily devoting effort to an OSS

∗ Corresponding author. Tel.: +1 205 348-9829.E-mail addresses: [email protected] (A. Bosu), [email protected]

J. Carver), [email protected] (R. Guadagno), [email protected] (B. Bassett),[email protected] (D. McCallum), [email protected] (L. Hochstein).

ttp://dx.doi.org/10.1016/j.jss.2014.03.061164-1212/© 2014 Elsevier Inc. All rights reserved.

© 2014 Elsevier Inc. All rights reserved.

project (Gutwin et al., 2004). The level of dedication a participanthas for the OSS project is strongly related to peer recognition (Xuand Jones, 2010). Therefore, gaining and maintaining reputation isa key factor in keeping an OSS project on track (Markus et al., 2000).When a participant is well recognized within the OSS community,his or her peers regard the participant’s project related opinionsmore carefully (Gacek and Arief, 2004).

Because gaining peer recognition is a major motivation for OSSparticipants and it influences OSS projects greatly, it is important tounderstand the peer recognition process within OSS communities.We define peer recognition as: the acknowledgement of a person’smerits or status by his or her peers. Before a person can acknowledgethe merits or status of a peer s/he must be aware of those mer-its or status. Therefore, it is important to understand how peersform opinions of each other in OSS communities. There is a largebody of research on peer recognition in the Psychology literature,where researchers use the term “impression formation” to describethe same concept. According to Kenny (1994), peer impression is

n open source organizations: A survey. J. Syst. Software (2014),

“the judgements that a person, called the perceiver, makes aboutanother person, called the target, where the target is a real person”.The formation of interpersonal impression primarily depends uponhow well the perceiver is acquainted with the target and upon the

Page 2: Peer impressions in open source organizations: A survey

ING ModelJ

2 tems a

pdpu

tucabMiaaotfpt(

ititasit

pSaShF

2

ciOethst

2

i1dbeawwihtwT

ARTICLESS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Sys

ersonality traits of those two individuals. Similarly, Moore (2007)efined impression formation as: “the process by which individualserceive, organize, and ultimately integrate information to formnified and coherent situated impressions of others”.

Generally, members of OSS communities are geographically dis-ributed, rarely or never meet face-to-face (FTF), and collaboratesing text-based tools over the Internet, i.e. computer-mediated-ommunication (CMC) (Guadagno and Cialdini, 2005; Jarvenpaand Leidner, 1998). Research has shown a marked differenceetween FTF and CMC regardless of the purpose. Specifically,cKenna and Bargh (2000) propose four domains in which social

nteraction via CMC differs from other more conventional inter-ction media: relative anonymity, reduced importance of physicalppearance, attenuation of physical distance, and greater controlver the time and pace of interactions. Due to those differences,he impression formation process between OSS participants is dif-erent than the impression formation process between co-locatedroject participants. However, there is a lack of knowledge abouthe impression formation between the participants of OSS projectsMarlow et al., 2013).

To better understand the formation and evolution of peermpressions in distributed OSS teams, we surveyed a broad spec-rum of OSS participants to discover: (1) how different forms of peermpressions develop in OSS communities, (2) the factors that affecthe impression formation process, (3) how peer impressions evolve,nd (4) the opinions of OSS participants about those peer impres-ions. In this study, we primarily focused on five dimensions of peermpressions: (1) productivity, (2) competency, (3) easy or difficulto work with, (4) perceived expertise, and (5) trustworthiness.

The remainder of the paper is organized as follows. Section 2resents the research questions and hypotheses for the survey.ection 3 describes the survey design. Section 4 explains the datanalysis process. Section 5 discusses the respondent demographics.ection 6 presents the results relative to the research questions andypotheses. Section 7 explains the threats to validity of the survey.inally, Section 8 concludes the paper.

. Research questions and hypotheses

Our study of the literature on reputation, communication andollaboration in OSS communities, identified eight important top-cs that can provide insight into the peer impression process inSS communities. For five of those topics, the literature presentednough evidence to pose definite hypotheses. For the other threeopics, we simply pose research questions, which may lead toypotheses for future study. This section provides a brief discus-ion of the literature to motivate each of the five hypotheses andhree research questions.

.1. Experiences working with other participants

The ‘craftsmanship model’ states that the pure joy of develop-ng software is a major motivation for OSS participants (Raymond,998). Studies have identified the most important reasons whyevelopers contribute to OSS projects to be: enjoyment, learningenefits (Hars and Ou, 2002; Lakhani and Wolf, 2005), and positivexperiences from participation (Xu and Jones, 2010). Conversely, ifn OSS participant’s experiences are continually negative, he or sheill eventually leave the project (Von Krogh et al., 2003). Therefore,e expect that participants in successful OSS projects will have pos-

tive experiences. Even so, it is likely that some OSS participants will

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

ave negative experiences. To better understand impression forma-ion, it is important to understand the OSS participants’ experiencesorking with their peers and what factors affect those experiences.

herefore, we pose the following research question:

PRESSnd Software xxx (2014) xxx–xxx

RQ1: What positive and negative experiences do OSS participantshave while working with their peers?

2.2. Perceived expertise

An OSS community member gains reputation primarily basedupon his or her consistent high quality contributions (Raymond,1998). Most OSS activities are highly knowledge-intensive andrequire a certain level of expertise (Von Krogh et al., 2003). There-fore, an OSS participant displays expertise through his or hercontributions to the project. The impression that people have aboutanother person’s expertise affects whether they trust that person’sopinions (Moorman et al., 1993). This finding is true both on and offline (Cialdini, 2008; Guadagno and Cialdini, 2005). The interactionbetween perceived expertise and interpersonal interaction leads tothe following hypothesis:

H1: An OSS participant considers his or her impression of a peer’sexpertise an important factor affecting their interactions with thatpeer.

2.3. Trust

Psychological research has demonstrated the importance oftrust in establishing online relationships (Green, 2007). Similarly,research on team performance suggests that a virtual team needsa solid foundation of mutual trust to enable effective collabora-tion (Jarvenpaa and Leidner, 1998; Holton, 2001; Peters and Manz,2007). Virtual teams cannot be effective without trust, becauseindividual members are not willing to take the risk that a teammember will act in his or her own self-interest, rather than theinterest of the team (Zand, 1972). Because OSS teams are a primeexample of virtual, online communities, we can hypothesize thefollowing:

H2: An OSS participant considers his or her level of trust of a peerimportant when interacting with that peer.

2.4. Losing mutual trust

OSS participants are quite diverse relative to age, race, nation-ality, and educational background (Ghosh et al., 2002; Lakhani andWolf, 2005). The participants also have diverse skills and inter-ests. The diversity often causes conflicts (Jensen and Scacchi, 2005),which may result in lost trust. Again, one participant may be veryenthusiastic but not as competent as another participant. Hence,his/her repeated failures may cause the project owners to loseconfidence in him/her. There may be other reasons that OSS par-ticipants lose mutual trust. Because the literature did not provideenough evidence to hypothesize the most important factors, thisresearch question seeks to identify those factors.

RQ2: Which factors influence OSS participants to lose trust in theirpeers?

2.5. Meeting in person

Most OSS community members are geographically distributed,rarely or never meet in person, and coordinate primarily via text-based communication tools (e.g., mailing list, Internet Relay Chat(IRC), repositories, and wikis). Because those communication toolscannot capture facial expressions and body language, it may be dif-ficult to understand and interpret the tone of the communication

n open source organizations: A survey. J. Syst. Software (2014),

(Peters and Manz, 2007). In addition, research has shown that vir-tual teams who use FTF meetings for team building and solvingcomplex issues were more effective than teams that did not useFTF meetings (Maznevski and Chudoba, 2000).

Page 3: Peer impressions in open source organizations: A survey

ING ModelJ

tems a

ACcidsnAsepcodw

2

tlbemncapeSftt

2

tcctnmcub

2

iihrfj

ARTICLESS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Sys

OSS participants often meet each other at conferences (e.g.,pacheCon, EclipseCon, PyCon, and MySQL AB conference).rowston et al. (2007) interviewed OSS participants at five OSSonferences to understand the impact of FTF meetings. The resultsndicated that meeting in person helps build social ties amongevelopers, which in turn facilitates better interactions. Morepecifically, one interviewee mentioned that s/he felt more con-ected to other participants after meeting with them the first time.nother interviewee mentioned that s/he was more comfortableending other participants email after meeting them (Crowstont al., 2007). These results are similar to results reported in thesychology literature that social cues (such as a little biographi-al information about a virtual colleague) facilitate the formationf positive impressions (Tanis and Postmes, 2003). The apparentifference between people who interact FTF compared with thoseho do not leads us to pose the following hypothesis:

H3: When OSS participants meet in person, their impressions ofeach other improve.

.6. Belief about peers’ impressions

Participation in OSS projects is highly visible because most ofhe activities occur via the Internet. OSS repositories and mailingist archives are open for public viewing. This visibility allows mem-ers of the community to monitor the performance and behavior ofach community member. Members are generally aware of whichembers do good work and which members violate the commu-

ity norms (Markus et al., 2000). This transparency should allowommunity members to form an accurate opinion of their peers’bilities. To maintain awareness between project members, OSSrojects use mailing lists, IRC channels, and commit logs (Gutwint al., 2004). OSS project hosting sites (i.e. GitHub, LaunchPad, andourceForge) provide participants with dashboards and activityeeds to help them track project plans and other members’ con-ributions (Treude and Storey, 2010). Thus, we can hypothesizehat:

H4: OSS participants believe that their peers have an accurateimpression of their abilities.

.7. Judging peer productivity

Raymond (1998) states that OSS communities have a ‘gift cul-ure’ by saying that “... social status is determined not by what youontrol but by what you give away”. An OSS community member’sontributions determine his or her identification and reputation inhe community (Roberts et al., 2006). Moreover, due to the virtualature of OSS collaboration, the differences in online vs. FTF com-unication (McKenna and Bargh, 2000), and the ease with which

ontributions can be monitored, we believe that project contrib-tions are the most reasonable way to judge a peer. To test thiselief, we hypothesize:

H5: OSS participants consider contributions of a peer to the projectas the most important factor when evaluating the productivity orcompetency of that peer.

.8. Judging a peer as easy or difficult to work with

Multiple factors can affect whether someone believes work-ng with a particular peer is easy or difficult. Factors related tondividual characteristics may include: creativity, responsibility,

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

onesty, and use of negative words (e.g., swear words, insults, oracial remarks). Because virtual community members are very taskocused (Guadagno and Cialdini, 2002), an OSS participant may alsoudge a peer based upon the quality of his or her work. For example,

PRESSnd Software xxx (2014) xxx–xxx 3

in OSS communities, a participant may judge a peer by his or hercode. The literature provided no conclusions about which particu-lar factor(s) might be more important for judging whether workingwith a peer would be easy or difficult. Therefore, to identify thesefactors, we posed the following question:

RQ3: What factors affect whether working with a peer on an OSSproject will be easy or difficult?

3. Survey design

To investigate the hypotheses and question posed in Section 2,we designed an online survey about communication, collaboration,peer evaluation and conflict resolution between OSS participants.Using our varied expertise (i.e. software engineering and psychol-ogy), we worked together to develop a set of 20 questions, of whicheight used a 5-point scale, four used multiple-choice answers,and the remaining eight used open-ended answers. For the mul-tiple choice and 5-point scale questions, we drew the answerchoices from the OSS literature (Gallivan, 2001; Hertel et al., 2003;Raymond, 1998; Roberts et al., 2006; Stewart and Gosain, 2006;Xu and Jones, 2010), our own experiences, discussions in OSS com-munity forums and examination of OSS mailing lists. This paperdescribes the results of the 15 questions that most directly relateto our hypotheses and questions. Appendix A provides a copy ofthe survey.

To distribute the survey, we identified 50 successful OSScommunities (i.e. at least 50,000 downloads) that had active devel-opment communities and used development mailing lists forcollaboration. Mailing lists are a critical communication tool forOSS communities (e.g., python-dev for the Python project). Becauseeach active developer subscribes to the development mailing list,these mailing lists reach the appropriate survey population. Toensure that our survey request email was distributed to the list, wecontacted the list administrators to obtain access/instructions forsending the survey request. In the survey request email, we spec-ified that only recipients who were actively participating in OSSprojects should consider responding to the survey. We subscribedto the mailing lists to verify that the survey request was sent out.Our survey request was posted to 48 of the 50 mailing lists duringthe second week of June 2011. We sent a reminder email to each listone week after the first email. When the rate of responses slowed,approximately two weeks later, we closed the survey. Overall, 115developers responded.

4. Data analysis

The survey produced three types of data. For the multiple choicequestions (nominal data) and the 5-point scale questions (ordinaldata), we used the non-parametric Chi-square and Mann–WhitneyU tests, respectively. Both tests are robust and do not require nor-mally distributed data. We used the standard alpha value of.05 forjudging significance. For the open-ended questions, we followed asystematic qualitative data analysis process to generate nominaldata that could be analyzed with the Chi-square test. Each authorhad his/her own role in data analysis. First, Bassett extracted thegeneral theme associated with each response. Next, Carver andHochstein, with Software Engineering expertise, and Guadagno andMcCallum, with Psychology expertise, analyzed these themes todevelop an agreed-upon coding scheme for each question.

Using these coding schemes, Bosu and Bassett worked indepen-dently to code the responses. After coding, they examined their

n open source organizations: A survey. J. Syst. Software (2014),

results to identify any discrepancies. They discussed those dis-crepancies and were able to resolve most of them. For the smallnumber of discrepancies they could not resolve, Carver providedthe tie-breaking vote after listening to the competing rationales.

Page 4: Peer impressions in open source organizations: A survey

ING ModelJ

4 tems a

5

qr

5

wr(pwtcmpttm

5

ofdfitfcfdw

5

neptvebavwp

5

hottio

6

s

ARTICLESS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Sys

. Respondent demographics

This section characterizes the sample based on responses to fiveuestions to help readers properly interpret the applicability of theesults.

.1. OSS projects represented

The respondents indicated their primary OSS project. Althoughe sent the survey to the mailing lists of 48 OSS projects, the

espondents listed 67 unique projects as their primary projectsTable 1). The reason 67 projects were represented in our sam-le could be because 74% of the respondents indicated that theyorked on multiple OSS projects. A participant who is a minor con-

ributor on one of the 48 projects solicited could also be a majorontributor to another OSS project. In addition, OSS participantsay subscribe to the email lists of other popular OSS projects sim-

ly to stay informed about the latest development activities withinhat community. The survey instructed the respondents to answerhe remaining questions based on their experiences on their pri-

ary project.

.2. Project role(s) of the respondents

Fig. 1 shows the percentage of respondents who perform vari-us roles (note that a respondent can perform multiple roles and inact 90% of the respondents reported multiple roles). Most respon-ents (80%) perform some types of maintenance activities (e.g.xing bugs, maintain infrastructure, package maintenance, bugriage, or general maintenance). About 75% of the respondents per-orm development activities (e.g., adding new features, committingodes or module integration). About 50% of the respondents per-ormed various other roles (e.g., reporting bugs, participating iniscussions, consultancy, system administration, and conductingebcasts).

.3. Voluntary vs. paid-participation

Many popular OSS projects are sponsored by commercial orga-izations (e.g., Google sponsors Chromium and Android). Somemployees of these sponsoring companies contribute to the OSSroject as part of their job. We call these participants paid par-icipants. We call the participants who are not paid to contributeolunteer participants. Because paid participants may have differ-nt motivating factors than volunteer participants, we expect theirehavior may differ. Interestingly, the sample was comprised ofpproximately the same number of each type of participant (52%olunteer participants and 48% paid). Using this even distributione analyze any differences in responses between the two partici-ant types in Section 6.

.4. Distribution of participants across organizations

Most respondents (71%) indicated that their projects wereighly distributed, i.e. contained contributors from more than fiverganizations. Conversely, only 13% of the respondents indicatedhat the projects were co-located, i.e. all participants came fromhe same organization. The remainder of the respondents (16%)ndicated that their projects were distributed across two to fiverganizations.

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

. Results

This section describes the analysis for each of the five hypothe-es and three research question described in Section 2. In addition

PRESSnd Software xxx (2014) xxx–xxx

to analyzing each hypothesis or question in isolation, we also ana-lyzed whether being a paid or volunteer participant affected theresults. Where the effect was present, we report the results.

6.1. RQ1: Experiences working with other participants

Using the qualitative analysis approach described in Section4, we split the responses into: positive experiences (65 responses,61%) and negative experiences (41 responses, 39%). We then furtherdivided the responses into sub-groups as shown in Fig. 2 (note thatthe total is greater than 100% because each respondent may haveprovided multiple answers). Overall, the majority of respondentshad positive experiences working with their peers, as evidenced bythe fact that the two most common codes are positive ones, and allbut one of the negative codes appear near the bottom of the dis-tribution. The respondents found their peers to be knowledgeable,respectful, and fun to work with. One respondent commented that:

“There’s folks that are really good at what they do and are a lot offun to work with.”

Other respondents found collaboration with other participantsto be an enjoyable and valuable learning experience. As one respon-dent put it:

“Most developers have a quite different set of skills and techno-logical background; this also allows me to learn new things as Icontribute to the project.”

It is important to examine the negative experiences in moredetail to help OSS communities take effective measures to elimi-nate or reduce the negative experiences. The most common reasonfor a negative experience was difficulties in collaboration and poormanagement, typically manifested as difficulty receiving a timelyresponse (9%). Some respondents were frustrated by the bureau-cratic decision-making process that made it difficult to obtain atimely and favorable decision about their proposed developmentdirections. Some respondents (about 8%) had good experienceswith the majority of the community but had problems getting alongwith a few people. For example, one respondent stated:

“Generally, the other developers have been attentive to the manyproblems that I have brought to their notice though there has beenone developer who has been over-sensitive to criticism and at timessomewhat non-cooperative and aggressive.”

Some participants may even leave a project due to the badbehavior of a peer. As another respondent noted:

“I was very happy working in the group for many years but decidedto leave due to the addition of one new developer who was argu-mentative, short and generally nasty to users who asked ratherbasic or dumb questions. I didn’t see why I should spend my freetime working on a project with such [a rude person].”

Some respondents (about 8%) found it very difficult to getinvolved with the community due to: lack of mentors, delayedresponse from project leaders, and a steep learning curve. For exam-ple, one respondent said,

“The process of getting involved is a bit tricky – they’re very openfolks, but getting into somewhere I can assist is difficult because ofan incredibly steep learning curve.”

As many OSS projects have participants separated by thousandsof miles and multiple time zones, some respondents (about 4%)mentioned that time-zone difference and distance were a hin-

n open source organizations: A survey. J. Syst. Software (2014),

drance to collaborating. If there is a large difference between twoparticipants’ time-zones, it is difficult to collaborate synchronously,which can lead to slower resolution of problems. Large geograph-ical distances may result in peers never meeting each other FTF

Page 5: Peer impressions in open source organizations: A survey

ARTICLE IN PRESSG ModelJSS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Systems and Software xxx (2014) xxx–xxx 5

Table 1Respondents’ primary OSS project.

Project # Resp. Project # Resp. Project # Resp. Project # Resp. Project # Resp.

Drupal 9 FreeBSD 7 Debian 6 OpenStack 5 Python 4Adium 3 Chromium 3 Jenkins 3 Macports 3 MediaWiki 3PHP 3 Asterisk 2 BugZilla 2 Eclipse Mylyn 2 Fedora 2GNU Emacs 2 PostGreSQL 2 Squid HTTP 2 Subversion 2 Ubuntu 2AlphaX 1 Apache Commons 1 Apache httpd 1 Apache Tomcat 1 argoUML 1arowboat.org 1 Bitweaver 1 Bro IDS 1 CCNx 1 Devnull 1DTC 1 Eclipse SDK 1 Escher CMS 1 Fink 1 FluidSynth 1Freeciv 1 FusionForge 1 GlassFish 1 GNS3 1 GNU Guile 1Mandriva 1 Maxilla 1 ModeShape 1 Mozilla 1 NetBSD 1One Click Orgs 1 OpenBIOS 1 OSM Potlatch 1 PSPP 1 R-Project 1Hibernate 1 HylaFAX 1 Innodb 1 Jboss AS 1 kiwix 1

taadropa

Mactex 1 Mahout 1 Rakudo

sbuild 1 Subversion-Edge 1 TortoiseHgweborf 1 Wordpress 1

o observe physical cues, which can lead to misunderstood emailsnd other textual communication. A few respondents (about 2%)lso observed conflicts between OSS participants resulting fromifferences in opinions and clashes of personalities. Although theespondents reported some negative experiences, the overall tonesf the responses indicated that the respondents found working witheers to be a positive experience. The following response provides

good summary of the overall experiences of the respondents:

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

“We’re from a wide variety of backgrounds and professions, andtherefore bring different perspectives to the project’s issues. Thiscan be both good and bad, but usually the good outweighs the bad.”

Fig. 1. Roles of the

Fig. 2. OSS participants’ experie

1 RISC gcc 1 Robot Framework 11 tunnelblick 1 VirtualBox 1

Fig. 3 shows that the ratio of positive to negative experiencesworking with peers was significantly dependent upon whetherthe respondent was a paid or volunteer participant (�2

1,106 =3.559, p = .05). In addition, Fig. 4 shows that specific types ofexperiences were reported with different frequencies by thetwo types of participants. Some experiences were more com-mon among paid participants while others were more commonamong volunteer participants. Of those experiences, the only one

n open source organizations: A survey. J. Syst. Software (2014),

that was significant was the frequency with which participantsfound their peers to be competent or dedicated (�2

1,106 = 4.771,p = .029).

respondents.

nces working with peers.

Page 6: Peer impressions in open source organizations: A survey

ARTICLE ING ModelJSS-9314; No. of Pages 12

6 A. Bosu et al. / The Journal of Systems a

Fig. 3. Experiences working with peers.

6

iempf1tp

Fi

Fig. 4. Details of experiences working with peers.

.2. H1: Impact of perceived expertise on peer interactions

Fig. 5 shows that 56% of the respondents considered theirmpression of the level of expertise of a peer as a very important orxtremely important factor during interaction with that peer. Theedian and the mode of the ratings were “4 – Very Important”. This

ositively skewed distribution is significantly different from uni-

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

orm (�24,112 = 70.468, p < .001). This result supports Hypothesis

. This result is also consistent with previous literature on virtualeams. Potter et al. (2000) highlighted the need for expertise as therimary reason that virtual collaborations form. Peters and Manz

ig. 5. Importance of peer’s: (a) perceived expertise and (b) trustworthiness fornteraction.

PRESSnd Software xxx (2014) xxx–xxx

(2007) and Kayworth and Leidner (2000) indicated that expertiseis an important factor for virtual team performance. Because OSScommunities are virtual teams (Gallivan, 2001), perception of peerexpertise should be an important factor affecting collaborating.

6.3. H2: Impact of trust on peer impression

Fig. 5 shows that 63% of the respondents considered their levelof trust in a peer to be either very important or extremely importantduring interaction with that peer. The mode and median of the rat-ings was “4 – Very Important”. This positively skewed distributionis significantly different from uniform (�2

4,112 = 54.125, p < .001).This result supports Hypothesis 2.

This result may be related to the importance of mutual trustin OSS communities found in the literature. Collaborative develop-ment efforts, such as those in OSS communities, cannot be effectivewithout mutual trust. Because contributions to the OSS projectfrom different participants have to work together without conflict,value of one participant’s contribution depends, in part, upon theefforts and contributions of the others (Stewart and Gosain, 2006).A developer also needs to trust that the project administrators willgive appropriate credit for his or her contribution. Low levels oftrust can result in higher developer turnover (Von Krogh et al.,2003). Other researchers have found that mutual trust betweenteam members positively affects the productivity of OSS projects(Xu and Jones, 2010; Stewart and Gosain, 2006; Lane et al., 2004).The importance of trust was also validated in a recent study thatshowed that before accepting a code change, OSS project ownersform an opinion about the trustworthiness of a code author basedon information in the author’s profile (Marlow et al., 2013).

6.4. RQ2: Losing trust in a peer

We asked the respondents to indicate which of the following11 factors caused them to lose confidence in a peer (Appendix A– Q14). Post-hoc, during the analysis process, we grouped the 11factors as follows:

1 Coding-related factors: Poor quality code, buggy features,introducing critical bugs, and codes violate design integrity.

2 Professional factors: Contributing code not consistent withproject goals, trying to sabotage project reputation, contributingto a rival project, and consistently missing deadlines.

3 Interpersonal factors: Telling lies, irresponsible behavior, andusing negative words in forums.

As shown in Fig. 6, the Coding-related factors were the topcauses for losing trust in a peer, followed by most of the Inter-personal factors.

To further understand trust, we asked whether it was possiblefor a peer to regain trust once confidence had been lost and whyor why not (A – Q15). Of the 80 respondents who answered thisquestion, 68 (85%) believed it was possible to regain lost trust, 7 (8%)believed it was not possible to regain lost trust and the remainderhad not experienced lost trust.

Of those who said it was possible to regain trust, 77% statedthat regaining trust is only possible through work. In addition, 16%stated that effective communication is required to regain trust. A

n open source organizations: A survey. J. Syst. Software (2014),

peer who has lost trust has to admit the fault, reverse the actionthat resulted in lost trust (if possible), demonstrate commitmentto the community, and produce quality work. Some respondentssaid that regaining trust is very difficult and takes a long time.

Page 7: Peer impressions in open source organizations: A survey

ARTICLE IN PRESSG ModelJSS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Systems and Software xxx (2014) xxx–xxx 7

Fig. 6. Factors that cause a peer to lose trust.

ting i

6

aiiitdrt

it

sas

pay his or her own expenses to attend project meetings or confer-ences. Volunteers are likely able to attend fewer conferences andproject meetings than paid participants. Moreover, small scale or

Fig. 7. Participants’ belief that mee

.5. H3: Effect of meeting in person on peer impression

The qualitative analysis of the 87 responses to the questionbout the effects of meeting a peer in person (A – Q11) resultedn the four responses shown in Fig. 7. The respondents overwhelm-ngly viewed meeting in person to have a positive impact on peermpression (71% vs. 11%). This result supports Hypothesis 3 andhe findings from Crowston et al. (2007). The following quotes helpescribe some of the reasons for the positive responses. First, manyespondents mentioned that meeting in person helps to humanizehe peer. For example:

“Meeting a person in real life allows one to identify with them asan individual and develop a rapport (or not!). Until then they aremerely a name, and one’s impression of them is solely based uponthe words they have written in communications (mail, IRC) andin code (patches, commits). I have found that such meetings aregenerally beneficial.”

Second, many respondents mentioned that meeting in personmproved communication because they could better understandhe messages. As one respondent said,

“We have a meeting every year, and it can make a lot of differenceto later be able to “hear” what their emails say as if they werespeaking. For example, one person turned out to be a very funnyperson in real life, but did not always understand that their jokesdo not go over well in written form. Having met them, it was easierto understand when they’re joking, and so get less upset when theysay something supposedly funny.”

Another respondent said,

“It makes further communication and interaction easier. I’ve dealtwith more people that I haven’t met and our interactions are per-fectly fine, but the ones I’ve met in person are a bit more loose.”

Third, some respondents mentioned that after meeting in per-on they felt a stronger sense of association and were able to assume

positive intent from email communications. As one respondent

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

aid,

“I’ve found that people (including myself) tend to do a better job ofassuming positive intent from people that they’ve met in person.”

n person affects their impressions.

Fig. 8 illustrates that the effects of meeting a teammate inperson were significantly different depending on the type of partic-ipant (�2

3,87 = 9.18, p = .027). Two key observations emerge fromthis data. First, approximately 20% of the volunteer participantsanswered “Don’t know/Have not met anyone,” compared with only2% of the paid participants. Second, 80% of the paid participantsanswered “Yes/Certainly,” compared with only 60% of the volunteerparticipants.

To better understand this result, we hypothesized that the paidparticipants would be more likely to work in the same office asother project members (who were likely also paid by the sameorganization to participate in the project). This co-location shouldhelp facilitate in person interactions. However, when we tested thishypothesis, we found no significant relationship between partici-pant type and project distribution (Section 5.4).

Therefore, we hypothesize an alternate explanation for thisresult. Companies that pay employees to participate in OSS projectsalso pay for those participants to attend project meetings andconferences, providing more opportunities to meet their peers inperson. Conversely, a volunteer participant will typically have to

n open source organizations: A survey. J. Syst. Software (2014),

Fig. 8. Opinions regarding meeting in person, paid participants vs. volunteers.

Page 8: Peer impressions in open source organizations: A survey

ARTICLE ING ModelJSS-9314; No. of Pages 12

8 A. Bosu et al. / The Journal of Systems a

umptsmtwmais

6

fb3tT

rnra

four Work Style factors appear, indicating moderate importance of

Fig. 9. Participants’ belief that peers have accurate impressions.

nsuccessful OSS projects, whose members tend to be volunteers,ight not be able to arrange a conference at all. For these reasons,

aid participants likely have more opportunities than volunteerso meet their peers. The survey data supports this conclusion as ithows that it is more common for volunteer participants to nevereet any peers in person. Approximately 20% of the volunteer par-

icipants said that they have never met anyone in person, comparedith only 2% of the paid participants. Because the paid participantseet their peers in person more frequently, more of them have

basis upon which to state that meeting in person affected theirmpression of their peers. We plan to test this hypothesis morepecifically in future surveys.

.6. H4: Accuracy of peer impression

The qualitative analysis of the 88 responses to A – Q10 resulted inour categories of answers. Fig. 9 shows that 55% of the respondentselieved their peers have an accurate impression of their abilities,3% were unsure, but leaning positive, and only 13% believed thatheir peers do not have an accurate impression of their abilities.hese results support Hypothesis 4.

Some specific quotes from the responses help to explain thisesult. Many respondents mentioned that because in OSS commu-ities everything is archived and open, participants who maintainegular collaboration with the community can easily get a sense of

participant’s ability. For example, one respondent said,

“Yes. A lot of the communication happens in the open *and* isarchived in public places. The openness of discussions means notonly that a lot of people can see what is going on, but also thateveryone is on the same boat regarding what happened, what was

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

discussed, what conclusions we reached, and what we are currentlydoing.”

Fig. 10. Importance ratings of the factors to de

PRESSnd Software xxx (2014) xxx–xxx

Some respondents mentioned that the collaborative develop-ment process is helpful for building peer impressions. For example,one respondent said,

“Yes. Code reviews are a daily metric.”

Conversely, some of the reasons given for inaccurate impres-sions include: not being able to spend enough time and not beingable to use full capabilities. As one respondent said,

“Probably not. There has never been a need to use my full capability.I just contribute in areas that are productive, and leave areas that Icould work in when there are other bodies who can do that work.”

Surprisingly, whether a participant was paid or volunteer didnot affect whether he/she thinks his/her peers have an accurateview of his/her abilities. We expected that more paid participantswould think their peers (who may be co-located in the same com-pany) have an accurate view of them than the volunteers (who aredistributed and may never meet) would. We will investigate thisresult further in future studies.

6.7. H5: Effect of project contributions on opinions aboutproductivity and competency

We asked the respondents to rate the importance, on a 5-pointscale, of the following 13 factors for judging whether a peer isproductive or competent (A – Q8).

1 Work style: Creativity, accuracy, response time and efficiency.2 Historical factors: A peer’s handle (i.e. screen name or loginId),

background and online profile.3 Project contribution factors: Quality and understandability of

committed code, quality and quantity of comments, critical fixesmade, important design decisions made, informal written com-munication and other work produced.

Fig. 10 shows the 13 factors sorted in increasing order accordingto the proportion of the top two ratings (Extremely important andVery important). The median for each factor can be identified byobserving which category the 50% line crosses through.

The Historical factors appear at the top of the chart, indicat-ing that the respondents found these factors to have relatively lowimportance for judging a peer to be productive. Next, three of the

n open source organizations: A survey. J. Syst. Software (2014),

these factors. The fourth Work Style factor, accuracy is the secondmost important factor overall. Finally, the Project Contributionfactors appear at the bottom of the chart, indicating them to be

cide a peer as productive or competent.

Page 9: Peer impressions in open source organizations: A survey

IN PRESSG ModelJ

tems and Software xxx (2014) xxx–xxx 9

tffbPHp

gacoot

p(fvieUp

dfItasj

tFtmorp

ARTICLESS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Sys

he most important overall. Even though accuracy is a Work Styleactor, it could also have been considered a Project Contributionactor because accuracy affects the quality of a participant’s contri-ution. These results suggest that overall, OSS participants considerroject Contribution to be more important than Work Style oristorical factors for judging the productivity or competency of aeer. These results support Hypothesis 5.

This result supports the findings of Marlow et al. (2013) thateneral coding ability, project relevant skills, and interaction stylere the most important factors for forming impressions about theompetency of a peer. The participants mentioned the use of vari-us cues to form impressions, such as: amount of activity, frequencyf commits, length of contribution, languages used, and discussionhreads.

The importance of Project Contribution factors is also not sur-rising. OSS communities are often described as ‘meritocracies’Fielding, 1999; Raymond, 1998; Schmidt and Porter, 2001) (theorm of governance where responsibilities are assigned to indi-iduals solely based upon their merits). Because committed codencludes the author names, the quality and understandability ofach participant’s code can be easily judged by his or her peers.nderstandable code also helps to facilitate collaboration. Unsur-risingly, this factor was chosen as the most important factor.

Furthermore, OSS participants typically discuss importantesign decisions and critical bugs in mailing lists and in IRC. There-ore, all developers who subscribe to the list or participate in theRC are able to judge the intellect of those developers that providehe solutions. Because the success and reputation of an OSS projectre critically dependent upon developers making good design deci-ions and fixing critical bugs, those factors are very important forudging whether a peer is productive.

Fig. 2 shows that approximately 19% of the respondents thoughtheir peers were “competent or dedicated” (RQ1 – Section 6.1).ig. 11 shows that respondents who found their peers “compe-ent or dedicated” gave significantly higher importance to the two

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

ost important factors in Fig. 10, “Quality and understandabilityf code” (Mann–Whitney U = 617.0, Z = −2.53, p = .011) and “Accu-acy” (Mann–Whitney U = 612.0, Z = −2.593, p = .01). Relative to theeer impression formation process, this result suggests that when

Fig. 12. Factors influencing how e

Fig. 13. Factors influencing the decision to

Fig. 11. Differences in distribution based on respondents’ impressions about peers.

a participant performs a significant task (e.g., developing a criti-cal module) with quality, understandability and accuracy, his/herpeers form a better impression.

6.8. RQ3: Factors affecting ease or difficulty of working together

The responses to Question 12 (Appendix A), shown in Fig. 12,illustrate that the respondents viewed five of the six factors as veryinfluential in judging how easy it is to work with a peer. Conversely,the responses to Question 13 (Appendix A), shown in Fig. 13, illus-trate how the respondents thought 10 of the 11 factors from RQ2affected their judgement of how difficult it was to work with a peer.In the same way as for RQ2, post-hoc we grouped these factors intothree categories. Not surprisingly, the results are similar to those forRQ2. The Coding-Related factors were the top reasons for decidingit was difficult to work with a peer, followed by most of the Inter-personal Factors. As a group, the Professional Factors were theleast influential. The similarity of the results for RQ2 and RQ3 sug-gests that OSS participants may start to lose confidence in a peerwhen they find it difficult to work with that peer. This hypothesis

n open source organizations: A survey. J. Syst. Software (2014),

will be tested in future studies.The fact that Coding-Related factors were the most important

is not surprising. Because OSS participants generally collaborate atthe code level, and often cannot have FTF meetings, understandable

asy it is to work with a peer.

judge a peer difficult to work with.

Page 10: Peer impressions in open source organizations: A survey

ING ModelJ

1 tems a

crqetp

rncipd

7

tc

7

tsbreiowtr

7

betoFT

vdact

7

rvtpfc

8

ps

ARTICLESS-9314; No. of Pages 12

0 A. Bosu et al. / The Journal of Sys

ode is of utmost importance. Moreover, due to the lack of formalequirements and documentation in most OSS communities, theuality and understandability of source code is very important forffective collaboration and future maintenance. Therefore, OSS par-icipants find it difficult to work with team members who writeoor quality code that has bugs or violates design integrity.

One interesting observation from Fig. 13 is that most of theespondents (70%) indicated that contributing to a rival project didot have a negative influence. It is possible that OSS participantsonsider working on a rival project to be advantageous by help-ng them become aware of the strong and weak features of theroject. The experiences from rival projects may also help guideevelopment directions.

. Threats to validity

As with any empirical study there are some threats to validityhat need to be discussed. This section is organized around threeommon types of validity threats.

.1. Internal validity

A threat to internal validity for our study is related to the selec-ion of study participants. We sent the survey requests only touccessful OSS project communities. Those communities might note a good representation of the overall OSS world. Although weequested only active OSS participants to respond, we could notnsure this fact. However, we believe there was no motivation fornactive participants to respond to the survey because we did notffer any financial incentive. While analyzing the qualitative data,e suspected that four respondents may have answered the ques-

ions carelessly (i.e. they provided random texts in the open-endedesponses). We excluded these responses from the analysis.

.2. Construct validity

First, we selected the factors to include in various questionsased on a reading of the literature and own personal experi-nces. The lists of factors may not be complete. To combat thishreat, we provided respondents with an “Others (Please specify)”ption. However, very few respondents selected the “Other” option.or those that did, no particular factor was mentioned frequently.herefore, we believe this threat is not significant.

Second, we did not perform a formal validity analysis of the sur-ey. Rather, we had the survey questions reviewed by survey andomain experts to ensure their understandability and to removeny potential bias. Because the results of the survey were largelyonsistent with findings from earlier studies, we do not believe thishreat is serious.

.3. External validity

We are not able to definitively establish that our sample is rep-esentative of the entire OSS population. While OSS communitiesary a lot based on product, participant type, community struc-ure, and governance, our sample represents a large number of OSSrojects. Even though we did not take into account most of thoseactors, we do have confidence that our sample is representative. Ofourse, the results may not be applicable to all OSS communities.

. Conclusion

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

This paper presents the results of a survey that explored theeer impression formation process in OSS communities. The resultshowed that the majority of survey respondents had positive

PRESSnd Software xxx (2014) xxx–xxx

experiences when collaborating with their peers. The respondentsconsider the perceived expertise and trustworthiness of their peersto be very important factors for interaction and impression forma-tion. Although meeting a peer FTF is infrequent, those meetings canpositively affect peer impression. Factors related to project con-tributions are the most important when forming an impression ofwhether a peer is productive and how easy or difficult it is to workwith a peer.

The psychology literature suggests that impression formationdepends on the familiarity with personal qualities. In OSS projects,participants are not aware of most of the personality traits of theirpeers. Therefore, OSS participants become very task-focused whenthey form impressions about a peer. The survey results indicatedthat the respondents placed the lowest emphasis on interpersonalfactors and the highest emphasis on project contributions whenforming impressions of their peers.

The introduction mentioned the four domains of CMC inter-action (i.e. relative anonymity, reduced importance of physicalappearance, attenuation of physical distance, and greater controlover the time and pace of interactions) proposed by McKennaand Bargh (2000). The current survey results address two of thosedomains. We found that collaboration between OSS participantsis relatively anonymous and they give the least importance to thebackground and profile of a peer. However, the results do not sup-port reduced importance of physical appearance. Although OSSparticipants rarely meet their peers in person, this does not implythat they do not want to meet in person or that meeting in per-son does not benefits them. In fact, the results show that meetingin person improves communication by helping participants betterunderstand the tone of a peer’s messages and helps to humanizetheir peers.

In this paper, we studied components of impression formationbetween OSS participants and found some evidences regardinghow those components form and evolve. The results providesome insight into the impression formation process between OSSpeers. Impression formation is different and complicated in virtualorganizations like an OSS community compared with traditionalorganizations. Yet, OSS participants consider those impressionsvery important for communication and collaboration. Therefore,we believe peer impressions will also affect the outcomes of OSSprojects. In the future, we plan to study how project outcomes areaffected by peer impressions and how to improve the impressionformation process in OSS organizations.

Acknowledgment

Bosu’s work was partially supported by the US National Sci-ence Foundation Grant No. 1322276. Carver’s work was partiallysupported by the US National Science Foundation Grants 1322276and 1156563. Basset’s work was partially supported by the USNational Science Foundation Grant No. 1156563. Any opinionsexpressed in this material are those of the authors and do notnecessarily reflect the views of the National Science Foundation.We also acknowledge anonymous reviewers for their insightfulsuggestions to improve this paper.

Appendix A. Survey questions

Q1. What open source project are you most actively involvedin?

n open source organizations: A survey. J. Syst. Software (2014),

Q2. What other open source project(s) have you actively workedon?

Q3. Please name or describe your roles on the open sourceproject? (e.g., add new code, report bugs, fix bugs, maintain project

Page 11: Peer impressions in open source organizations: A survey

ING ModelJ

tems a

ie

s

o

r

©©

©

©

©

hfti

sa4

y

o

wt2t

ARTICLESS-9314; No. of Pages 12

A. Bosu et al. / The Journal of Sys

nfrastructure, make strategic decisions about direction of projects,tc.)

Q4. Please briefly provide your thoughts on the quality of theoftware

Q5. Please briefly provide your thoughts about working withther developers on the project

Q6. Do you contribute to the open source project during youregular job, or only in your spare time?

I contribute during my regular job (getting paid) I only contribute in my spare time (volunteer)

Q7. How are the project contributors distributed?

Contributors are roughly evenly distributed across many (>=5)different organizations

Most contributors come from a small number (<5) of organiza-tions

Most contributors are members of one organization

Q8. When deciding if a teammate is productive or competent,ow important are each of the following factors? Please rate each

actor on a 1–5 scale (1 = Not at all important, 2 = Slightly impor-ant, 3 = Moderately important, 4 = Very important, 5 = Extremelymportant)

CreativityAccuracyResponse timeEfficiencyTheir handleTheir backgroundTheir online profileQuality and understandability of the code committed to reposi-toryQuality and quantity of comments in their codeOther work products produced (i.e. design documents, test cases)Informal written communication (i.e. post to mailing lists, IRCchats)Critical fixesImportant design decisions madeOthers(Please specify)

Q9. Thinking about your interactions with your teammates, on acale of 1–5 how important is each of the following factors? (1 = Nott all important, 2 = Slightly important, 3 = Moderately important,

= Very important, 5 = Extremely important).

Your impression of their level of expertiseHow much you trust them

Q10. Do you think your teammates have an accurate view ofour abilities? Why or why not?

Q11. Does meeting a teammate in person affect your impressionf them? Explain

Q12. Think of a time when you found a teammate easy toork with. How important were the following factors in deciding

hat teammate was easy to work with? (1 = Not at all important, = Slightly important, 3 = Moderately important, 4 = Very impor-ant, 5 = Extremely important)

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

CreativeResponsibleGood coding styleWilling to accept challenges

PRESSnd Software xxx (2014) xxx–xxx 11

Developed good quality featuresOthers(Please specify)

Q13. Think of a time when you found a teammate difficultto work with? Which factors influenced your decision? (1 = Notat all influential, 2 = Slightly influential, 3 = Moderately influential,4 = Very influential, 5 = Extremely influential)

Telling liesTrying to avoid responsibilitiesConsistently missing deadlinesPoor quality codeBuggy featuresCodes violating design integrityTrying to sabotage project reputationContributing to a rival projectUsing negative words in forums (e.g., swear words, insults, racialremarks, etc.)Contributing code that is not consistent with project goalsOthers(Please specify)

Q14. Have you ever lost confidence in a teammate to the pointwhere you would not trust their opinions (e.g., on design decisions),or the quality of their work products (e.g., code produced)? Whatfactors that caused you to lose this confidence? (Please check allapplicable)

� Telling lies� Consistently missing deadlines� Poor quality code� Introducing critical bug� Buggy features� Code violates design integrity� Trying to sabotage project reputation� Contributing to a rival project� Irresponsible behavior� Using negative words in forums� Contributing code that is not consistent with project goals

Q15. Is it possible for a teammate to regain trust once they havelost your confidence? If so, how?

References

Cialdini, R.B., 2008. Influence: Science and Practice, 5th ed. Allyn and Bacon, Boston,MA.

Crowston, K., Howison, J., Masango, C., Eseryel, U.Y., 2007. The role of face-to-face meetings in technology-supported self-organizing distributed teams. IEEETrans. Prof. Commun. 50 (3), 185–203.

Evans, P., Wolf, B., 2005. Collaboration rules. Harvard Bus. Rev. 83 (7), 96–104.Fielding, R.T., 1999. Shared leadership in the apache project. Commun. ACM 42 (4),

42–43.Gacek, C., Arief, B., 2004. The many meanings of open source. IEEE Softw. 21 (1),

34–40.Gallivan, M.J., 2001. Striking a balance between trust and control in a virtual orga-

nization: a content analysis of open source software case studies. Inform. Syst.J. 11 (4), 277–304.

Ghosh, R.A., Glott, R., Krieger, B., Robles, G., 2002. Free/Libre and OpenSource Software: Survey and Study. Technical Report. International Insti-tute of Infonomics, University of Maastricht, Maastricht, The Netherlandshttp://www.flossproject.nl/report/FLOSSFinal4.pdf

Green, M.C., 2007. Trust and social interaction on the Internet. In: Oxford Handbookof Internet Psychology. Oxford University Press, Oxford, UK.

Guadagno, R., Cialdini, R., 2002. Online persuasion: an examination of gender differ-ences in computer-mediated interpersonal influence. Group Dyn. Theory Res.Pract. 6 (1), 38–51.

Guadagno, R., Cialdini, R., 2005. Online persuasion and compliance: Social influence

n open source organizations: A survey. J. Syst. Software (2014),

on the Internet and beyond. In: Amichai-Hamburger, Y. (Ed.), The social net:Human behavior in cyberspace. Oxford University Press, New York, pp. 91–113.

Gutwin, C., Penner, R., Schneider, K., 2004. Group awareness in distributed soft-ware development. In: Proceedings of the 2004 ACM Conference on ComputerSupported Cooperative Work, CSCW ‘04, pp. 72–81.

Page 12: Peer impressions in open source organizations: A survey

ING ModelJ

1 tems a

H

H

H

J

J

K

K

L

L

M

M

M

M

M

M

P

P

RR

R

S

S

T

T

ARTICLESS-9314; No. of Pages 12

2 A. Bosu et al. / The Journal of Sys

ars, A., Ou, S., 2002. Working for free? motivations for participating in open-sourceprojects. Int. J. Electr. Commerce 6 (3), 25–39.

ertel, G., Niedner, S., Herrmann, S., 2003. Motivation of software developers in opensource projects: an internet-based survey of contributors to the Linux kernel.Res. Policy 32 (7), 1159–1177.

olton, J.A., 2001. Building trust and collaboration in a virtual team. Team Perform.Manage. 7 (3/4), 36–47.

arvenpaa, S.L., Leidner, D.E., 1998. Communication and trust in global virtual teams.J. Comput. Mediat. Commun. 3 (4), 791–815.

ensen, C., Scacchi, W., 2005. Collaboration, leadership, control, and conflict nego-tiation and the netbeans.org open source software development community.In: Proceedings of the 38th Annual Hawaii International Conference on SystemSciences. HICSS ‘05, 196b.

ayworth, T., Leidner, D., 2000. The global virtual manager: a prescription for suc-cess. Eur. Manage. J. 18 (2), 183–194.

enny, D.A., 1994. Interpersonal Perception: A Social Relations Analysis. GuilfordPress, New York.

akhani, K.R., Wolf, R.G., 2005. Why hackers do what they do: Understanding moti-vation and effort in free/open source software projects. In: Feller, J., Fitzgerald,B., Hissam, S., Lakhani, K. (Eds.), Perspectives on Free and Open Source Software.MIT Press, Boston, MA.

ane, M.S., Glen, V., Basnet, P., 2004. Trust in virtual communities involved infree/open source projects: an empirical study. In: Proceedings of the 15th Aus-tralasian Conference on Information Systems. ACIS’04.

arkus, M.L., Manville, B., Agres, E., 2000. What makes a virtual organization work?Sloan Manage. Rev. 42 (1), 13–26.

arlow, J., Dabbish, L., Herbsleb, J., 2013. Impression formation in online peer pro-duction: activity traces and personal profiles in GitHub. In: Proceedings of the2013 Conference on Computer Supported Cooperative Work, CSCW ‘13, pp.117–128.

aznevski, M.L., Chudoba, K.M., 2000. Bridging space over time: global virtual teamdynamics and effectiveness. Org. Sci. 11 (5), 473–492.

cKenna, K.Y., Bargh, J.A., 2000. Plan 9 from cyberspace: the implications of theinternet for personality and social psychology. Pers. Soc. Psychol. Rev. 4 (1),57–75.

oore, C.D., 2007. Impression Formation. In: Ritzer, George (Eds.), Blackwell ency-clopedia of Sociology, Vol. 2. Blackwell Publishing, NJ.

oorman, C., Deshpande, R., Zaltman, G., 1993. Factors affecting trust in marketresearch relationships. J. Market. 57 (1), 81–101.

eters, L.M., Manz, C.C., 2007. Identifying antecedents of virtual team collaboration.Team Perform. Manage. 13 (3/4), 117–129.

otter, R.E., Cooke, R.A., Balthazard, P.A., 2000. Virtual team interaction: assessment,consequences, and management. Team Perform. Manage. 6 (7/8), 131–137.

aymond, E., 1998. Homesteading the noosphere. First Monday 3 (10-5).aymond, E., 1999. The cathedral and the bazaar. Knowledge Technol. Policy 12 (3),

23–49.oberts, J.A., Hann, I.H., Slaughter, S.A., 2006. Understanding the motivations, par-

ticipation, and performance of open source software developers: a longitudinalstudy of the apache projects. Manage. Sci. 52 (7), 984–999.

chmidt, D.C., Porter, A., 2001. Leveraging open-source communities to improvethe quality performance of open-source software. In: Proceedings of the FirstWorkshop on Open-Source Software Engineering.

tewart, K.J., Gosain, S., 2006. The impact of ideology on effectiveness in open source

Please cite this article in press as: Bosu, A., et al., Peer impressions ihttp://dx.doi.org/10.1016/j.jss.2014.03.061

software development teams. MIS Quart. 30 (2), 291–314.anis, M., Postmes, T., 2003. Social cues and impression formation in CMC. J. Com-

mun. 53 (4), 676–693.reude, C., Storey, M., 2010. Awareness 2.0: staying aware of projects, develo-

pers and tasks using dashboards and feeds. In: Proceedings of the ACM/IEEE

PRESSnd Software xxx (2014) xxx–xxx

32nd International Conference on Software Engineering, ICSE ‘10, pp. 365–374.

Von Krogh, G., Spaeth, S., Lakhani, K.R., 2003. Community, joining, and specializationin open source software innovation: a case study. Res. Policy 32 (7), 1217–1241.

Xu, B., Jones, D.R., 2010. Volunteers’ participation in open source software develop-ment: a study from the social-relational perspective. SIGMIS Database 41 (3),69–84.

Zand, D.E., 1972. Trust and managerial problem solving. Admin. Sci. Quart. 17 (2),229–239.

Amiangshu Bosu is a PhD student in the Department of Computer Science at theUniversity of Alabama, and a member of the Software Engineering Research Group.He received a Masters degree from the University of Alabama in 2012, and a Bache-lors degree from the Bangladesh University of Engineering and Technology in 2006.His research interests lie in the areas of empirical software engineering, peer codereview, peer impression formation, and open source software.

Jeffrey C. Carver earned his PhD degree in Computer Science from the University ofMaryland. He is an Associate Professor in the Department of Computer Science atthe University of Alabama. His main research interests include empirical softwareengineering, software quality, software engineering for computational science andengineering, software architecture, human factors in software engineering and soft-ware process improvement. He is a Senior Member of the IEEE Computer Societyand a Senior Member of the ACM. Contact him at [email protected].

Rosanna Guadagno received her Ph.D. in Social Psychology from ArizonaState University. Her master’s and dissertation examined gender differences intechnology-mediated persuasion. Her postdoctoral work, at UCSB’s Research Centerfor Virtual Environments and Behavior, examined social influence in immersive vir-tual environments. At the University of Alabama, she founded and still directs UA’sOnline Social Influence Lab. Currently, she is serving as a Program Director at NSFin Social Psychology and Secure and Trustworthy Cyberspace. Her research inter-ests focus on the confluence of three main areas: Social Influence and Persuasion,Mediated-Communication, and Gender Roles.

Blake Bassett is a PhD student in the Department of Computer Science at the Uni-versity of Illinois Urbana-Champaign, and a member of the Automated SoftwareEngineering Research group. He received a Masters degree from the University ofAlabama in 2013 with a thesis involving optimizations to software feature locationtechniques using text retrieval. His interests lie in the areas of software engineering,programming languages, and human–computer interaction.

Debra McCallum is a Senior Research Social Scientist and Director of the Institutefor Social Science Research at the University of Alabama. She is also Director of theInstitute’s survey research center, known as the Capstone Poll. McCallum receivedher Ph.D. in Social Psychology from the University of North Carolina at Chapel Hilland has conducted research in a wide variety of settings. Her work includes evalu-ations of education and community intervention programs and research on socialissues, such as career choices related to STEM fields, social-psychological aspectsof health behavior and outcomes, effects of poverty, and safety and well-being ofchildren and youth.

Lorin Hochstein is a Lead Software Engineer at SendGrid Labs. He received hisPh.D. in Computer Science at the University of Maryland in 2006. Previously, he

n open source organizations: A survey. J. Syst. Software (2014),

was an Assistant Professor in the Department of Computer Science at Engineer-ing at the University of Nebraska - Lincoln, a Computer Scientist at University ofCalifornia’s Information Sciences Institute, and Lead Architect for Cloud Servicesat Nimbis Services. He has a lifelong interest in improving software developmentproductivity.


Recommended