Using Gamification to Orient and Motivate Students to Contribute to OSS projects
Guilherme C. Diniz*, Marco A. Graciotto Silva* * Computing Department,
Federal University of Technology – Paraná Campo Mourão, PR, Brazil
[email protected], [email protected]
Marco A. Gerosa#, Igor Steinmacher*,#
# School of Informatics, Computing, and Cyber Systems Northern Arizona University
Flagstaff, AZ, USA [email protected], [email protected]
Abstract—Students can benefit from contributing to Open Source Software (OSS), since they can enrich their portfolio and learn with real world projects. However, sometimes students are demotivated to contribute due to entrance barriers. On the other hand, gamification is widely used to engage and motivate people to accomplish tasks and improve their performance. The goal of this work is to analyze the use of gamification to orient and motivate undergraduate students to overcome onboarding barriers and engage to OSS projects. To achieve this goal, we implemented four gaming elements (Quests, Points, Ranking, and Levels) in GitLab and assessed the environment by means of a study conducted with 17 students, within a real OSS project (JabRef). At the end of the study, the students evaluated their experience through a questionnaire. We found that the Quest element helped to guide participants and keep them motivated and points helped by providing feedback on students’ performed tasks. We conclude that the gamified environment oriented the students in an attempt to make a contribution and that gamification can motivate and orient newcomers’ to engage to OSS projects.
Keywords - newcomers; open source software; engagement; gamification; motivation; students
I. INTRODUCTION
The Open Source Software (OSS) model has become an important driving force in today’s software development, resulting in many prominent projects that are used extensively through the entire development stack, from kernels to sophisticated end-user applications. Therefore, it is no surprise that the OSS movement attracts a large, globally distributed community of volunteers. In addition, the number of job vacancies valuing open source knowledge and experience has been rising on a regular basis [1]. This fact motivated students to contribute to OSS projects, and teachers to bring OSS practices and process to the classroom and foster students contributing to OSS projects [1–3].
Exposing students to OSS projects is beneficial both to OSS communities and students. From the perspective of the communities, more professionals will be in touch with OSS, ultimately leading to a higher number of contributions, since students are potential OSS contributors. They usually have the basic theoretical knowledge to contribute to a project, but lack practical skills and knowledge about the underlining technologies. From the perspective of the students, contributing is beneficial since working on OSS projects
enables them to learn real-world skills, attitudes, and experiences [2], [3], which might increase their confidence when applying for industry jobs. One can claim that this is also beneficial since the future workforce of software developers is being prepared practicing in a real scenario.
However, it is already known that newcomers, including the students, face many barriers while attempting to contribute to OSS [4–6]. Some barriers they face include orientation issues that can potentially demotivate newcomers from placing their first contribution. Many recent studies had being conducted aiming to engage and motivate newcomers to OSS projects [4], [7–9].
Gamification, which consists on the application of game elements in non-gaming contexts [10], is calling attention recently since it has been successfully applied to motivate and engage contributors of online and collaborative communities and business [11], [12]. These gaming elements had been successfully applied to different domains, including Software Engineering [12–14] and learning [15], [16]. From the best of our knowledge, there are no initiatives or studies focusing on the use of gamification to engage and support students or newcomers to overcome barriers to join OSS projects.
Therefore, the goal of this work is to propose and analyze the use of gamification elements to motivate and support undergraduate students to overcome the orientation barriers to contribute to OSS projects. To achieve this goal, we have selected four game design elements to help students: Rankings, Quests, Points, and Levels [14], [16], [17]. For each of these elements, we defined a set of rules to describe the operation of the gamified environment and implemented them on GitLab1. To assess the environment, we conducted a study with 17 undergraduate students, who evaluated their experience by means of a questionnaire. This was a preliminary study towards a gamified environment that can be used to foster contributions from students, benefiting both projects and students. The contributions of this study include presenting a gamified version of GitLab; assessing the use of Rankings, Points, Quests, and Levels to motivate and support newcomers to contribute to OSS projects; and proving insights to community building teams on how to use gamification elements to guide newcomers.
1 http://www.gitlab.com
use of game design elements in nonGamifgreat impact in different areasmeansStackOverpoints that can be earned by the users when they perform some tasks.
of intereststudies used newsfeeds and ranking to gamify a version control system to foMelo et alelements to encouragedevelopers. gamrewards,and the quality of their contributions. Snipes et al. mottecscoring systems in the context of software development.
and analyze the use of gamificatEducation. by gamifying aincrease student involvement documentet al.training people on defecthat are agile prpatterns
literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspecgamification is mainly used with the purpose of improving teaching and learning. of gamification previously evidenced barriers faced [29]contribution. inin OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS claimpr
elements ove
II. R
Gamification is defined by use of game design elements in nonGamification has being growing in the last years, presenting great impact in different areasmeans of engaging and motivatingStackOverflow and Wikipedia make upoints that can be earned by the users when they perform some tasks.
In software engineering, of interest recentlystudies [14], [16]used newsfeeds and ranking to gamify a version control system to foster more commits from students. Similarly, Melo et al. [21]elements to encouragedevelopers. Lotufogamified bug tracking system, based on reputation and rewards, would and the quality of their contributions. Snipes et al. [23]motivate developers to use more efficient development techniques, and Bacon et al.scoring systems in the context of software development.
In addition, and analyze the use of gamificatEducation. For example, by gamifying aincrease student involvement documentation, bet al. [26] presenttraining people on defecthat have taken are agile processespatterns [15].
From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspecgamification is mainly used with the purpose of improving teaching and learning. of gamification previously evidenced barriers faced [29], avoiding demotivation while contribution. Finitiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS claim that this work will presenting a new way of engaging students in OSS.
To achieve our goalelements to engage overcoming barriers to contribute to OSS projects, we
RELATED WORK ON
SOFTWARE
amification is defined by use of game design elements in non
cation has being growing in the last years, presenting great impact in different areas
of engaging and motivatingflow and Wikipedia make u
points that can be earned by the users when they perform
In software engineering, gamification recently [20], and already
[14], [16]. Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control
ster more commits from students. Similarly, [21] proposed the use of a newsfeed and ranking
elements to encourage collaboration and commitmentLotufo et al.
fied bug tracking system, based on reputation and would encourage teams to increase th
and the quality of their contributions. [23] investigated
vate developers to use more efficient development niques, and Bacon et al. [24]
scoring systems in the context of software development.In addition, there are some studies that report experiences
and analyze the use of gamificatFor example, Sheth et al.
by gamifying a software engineering course increase student involvement
tion, bug reportingpresented a gamification proposal focused on
training people on defect trackinghave taken advantage of gamification to foster learning
cesses [27], mutant testing
From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspective of software engineering, we see that gamification is mainly used with the purpose of improving teaching and learning. In contrastof gamification as a strategy to previously evidenced barriers faced
, avoiding demotivation while From the best of our knowledge, there is no
itiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS
that this work will add to the statesenting a new way of engaging students in OSS.
III.
To achieve our goal of assessing the use of gamification engage and support undergraduate students
coming barriers to contribute to OSS projects, we
ORK ON GAMIFICATION AND
OFTWARE ENGINEERING
amification is defined by Deterding et al. use of game design elements in non-game contexts
cation has being growing in the last years, presenting great impact in different areas [11], [18], [19]
of engaging and motivating people [11]flow and Wikipedia make use of b
points that can be earned by the users when they perform
gamification has become a topic , and already was target of
Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control
ster more commits from students. Similarly, proposed the use of a newsfeed and ranking
collaboration and commitmentet al. [22] investigated
fied bug tracking system, based on reputation and encourage teams to increase th
and the quality of their contributions. Also worth to mention, the use of a gamified system to
vate developers to use more efficient development [24] investigated
scoring systems in the context of software development.there are some studies that report experiences
and analyze the use of gamification on Software Engineering Sheth et al. [12], [25]
software engineering course increase student involvement on
ug reporting, and testing activitiesa gamification proposal focused on
t tracking. Other examples of advantage of gamification to foster learning
, mutant testing [28]
From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From
tive of software engineering, we see that gamification is mainly used with the purpose of improving
In contrast, in this paper we make use as a strategy to help students
previously evidenced barriers faced to join OSS , avoiding demotivation while they attempt to
rom the best of our knowledge, there is no itiative to gamify an OSS environment to engage students
in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS for the society
add to the statesenting a new way of engaging students in OSS.
METHOD
of assessing the use of gamification and support undergraduate students
coming barriers to contribute to OSS projects, we
AMIFICATION AND
NGINEERING
Deterding et al. [10] as “the game contexts
cation has being growing in the last years, presenting [11], [18], [19] , mainly as a
[11]. For example, se of badges and
points that can be earned by the users when they perform
has become a topic was target of secondary
Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control
ster more commits from students. Similarly, proposed the use of a newsfeed and ranking
collaboration and commitment investigated whether
fied bug tracking system, based on reputation and encourage teams to increase the frequency
Also worth to mention, the use of a gamified system to
vate developers to use more efficient development investigated the adoption of
scoring systems in the context of software development. there are some studies that report experiences
Software Engineering [12], [25] showed that
software engineering course it is possible development, activities. Thomas
a gamification proposal focused on examples of areas
advantage of gamification to foster learning [28], and design
From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From the
tive of software engineering, we see that gamification is mainly used with the purpose of improving
n this paper we make use students overcoming
to join OSS projects [5], attempt to place their
rom the best of our knowledge, there is no itiative to gamify an OSS environment to engage students
in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS on the academic
for the society, we add to the state-of-the-art by
senting a new way of engaging students in OSS.
of assessing the use of gamification and support undergraduate students
coming barriers to contribute to OSS projects, we
as “the game contexts.”
cation has being growing in the last years, presenting , mainly as a
. For example, and
points that can be earned by the users when they perform
has become a topic secondary
Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control
ster more commits from students. Similarly, proposed the use of a newsfeed and ranking
of whether a
fied bug tracking system, based on reputation and e frequency
Also worth to mention, the use of a gamified system to
vate developers to use more efficient development the adoption of
there are some studies that report experiences Software Engineering
showed that it is possible to
development, Thomas
a gamification proposal focused on areas
advantage of gamification to foster learning design
From software engineering perspective, we see that the literature brings approaches to improve existing processes
the tive of software engineering, we see that
gamification is mainly used with the purpose of improving n this paper we make use
coming [5],
place their rom the best of our knowledge, there is no
itiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.
academic , we
art by
of assessing the use of gamification and support undergraduate students
coming barriers to contribute to OSS projects, we
followed the research method depicted in Figure 1 and steps are
A. Step 1: Defining the gaming elements
The first step be suitable for supporting our goal. Based on the knowledge about the orientation band onengineeselected the following four elements, since they are in line with our goals, points, and ranking.
Questsdefined goal that needs to be accomplishing the tasks may be, for example, points, level change or connected to the be unlocked, enabling the users to acquire new skills. context of motivate andbarriers follow
Levelsto unlock specific and contextualizedCsikszentmihaly keep userthe level is too hardmotivation and stop playing. applied in conjunction on a given step before moving ahead on the contribution process.
Pointswork [13], [14], [16]progress to the users. Acquiring points means that the actions performed by the users are in accordance with what was expected. Point cperformed by the user. For example, in OSS, a user can earn points by having a solving a project member. users motivated and to orientation barriers, more specifically, providing feedback on their actions.
Rankingcompare to other players. different levels (global, regional, per projectare usuallysorted list, with the best players at the top. According to
followed the research method depicted in Figure 1 and steps are explained in the following subsections
Figure 1.
Step 1: Defining the gaming elements
The first step is defining the be suitable for supporting our goal. Based on the knowledge about the orientation b
on studies onengineering [14], [17]selected the following four elements, since they are in line with our goals, and are the points, and ranking.
Quests are assignments or tasks with a clear and welldefined goal that needs to be accomplishing the tasks may be, for example, points, level change or external rewarconnected to the questsbe unlocked, enabling the users to acquire new skills. context of this work
vate and support newcomers obarriers [4]. Quests can be used as a stepfollow to achieve the goal of placing a contribution.
Levels groups different quests and tto unlock specific and contextualized
szentmihaly [30]keep users in a set of challenges aligned the level is too hard motivation and stop playing. applied in conjunction on a given step before moving ahead on the contribution process.
Points is the most [13], [14], [16]
gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was expected. Point can be earned by means of different actions performed by the user. For example, in OSS, a user can earn points by having a pullsolving a quest, or working on a review
ject member. In this work, users motivated and to
tation barriers, more specifically, providing back on their actions.
Ranking enable users to follow their performance and compare to other players. different levels (global, regional, per projectare usually closely related to sorted list, with the best players at the top. According to
followed the research method depicted in Figure 1 and in the following subsections
Figure 1. Overall method followed
Step 1: Defining the gaming elements
defining the gabe suitable for supporting our goal. Based on the knowledge about the orientation barriers from our previous work
on gamification applied to software [14], [17] and to learning context
selected the following four elements, since they are in line and are the widely
are assignments or tasks with a clear and well
defined goal that needs to be performedaccomplishing the tasks may be, for example, points, level
rewards [13]. The quests, since more challenging
be unlocked, enabling the users to acquire new skills. this work, we want to assess how
support newcomers oQuests can be used as a step
the goal of placing a contribution. different quests and t
to unlock specific and contextualized[30] describes the
set of challenges aligned or too easy, the player can easily lose
motivation and stop playing. In our context, applied in conjunction to Quests on a given step before moving ahead on the contribution
is the most applied element [13], [14], [16]. It provide
gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was
an be earned by means of different actions performed by the user. For example, in OSS, a user can earn
pull-request acceptedor working on a review
In this work, Pointsusers motivated and to support them overcoming
tation barriers, more specifically, providing back on their actions.
enable users to follow their performance and compare to other players. It creates a sense of comdifferent levels (global, regional, per project
closely related to Pointssorted list, with the best players at the top. According to
followed the research method depicted in Figure 1 and in the following subsections.
Overall method followed
Step 1: Defining the gaming elements
game elements be suitable for supporting our goal. Based on the knowledge
arriers from our previous work gamification applied to software and to learning contexts
selected the following four elements, since they are in line widely used: quests, levels,
are assignments or tasks with a clear and wellperformed. The rewards of
accomplishing the tasks may be, for example, points, level The level element
, since more challenging be unlocked, enabling the users to acquire new skills.
we want to assess how support newcomers overcoming orientation
Quests can be used as a step-by-step that the goal of placing a contribution. different quests and tasks, enabling users
to unlock specific and contextualized set of actions. describes the flow theory as a way to
set of challenges aligned to their skillset. If , the player can easily lose
In our context, Level to make students focus
on a given step before moving ahead on the contribution
applied element as showed in previous . It provides feedback and show
gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was
an be earned by means of different actions performed by the user. For example, in OSS, a user can earn
accepted, helping otheror working on a review suggested by a
Points were chosen to support them overcoming
tation barriers, more specifically, providing
enable users to follow their performance and It creates a sense of com
different levels (global, regional, per project, etc.). Points, and are presented as a
sorted list, with the best players at the top. According to
followed the research method depicted in Figure 1 and its
me elements that would be suitable for supporting our goal. Based on the knowledge
arriers from our previous work [4], [5] gamification applied to software
s [16], we selected the following four elements, since they are in line
quests, levels,
are assignments or tasks with a clear and well-The rewards of
accomplishing the tasks may be, for example, points, level element is very
, since more challenging quests can be unlocked, enabling the users to acquire new skills. In the
we want to assess how Quests vercoming orientation
step that users the goal of placing a contribution.
asks, enabling users set of actions.
as a way to their skillset. If
, the player can easily lose Levels were
to make students focus on a given step before moving ahead on the contribution
in previous and shows
gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was
an be earned by means of different actions performed by the user. For example, in OSS, a user can earn
other users, suggested by a chosen to keep
support them overcoming tation barriers, more specifically, providing
enable users to follow their performance and It creates a sense of competition, in
etc.). Rankings presented as a
sorted list, with the best players at the top. According to
Deterding et al. to gamify an environment. Rankingpre
B.
cocan be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task, stidefinition is related to elements chosen. Figure how the rules and elements are interrelated.
the system. Each given amount of rewarded only oncenumber of times (or have its reward reduced). The player can also choose taccording to the user one or more can user. addco
accordance to the steps newcomerplace a contributionthe community.following the flow FLOSScoach portal. in
C.
of GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issuand wikis. envdashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the
Deterding et al. to gamify an environment. Ranking was prestige among the colleagues.
B. Step 2: Defining the rules
Rules, when combined, make up a set of provisions that condition the execution of a game can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task, stimulating and keeping the players engaged.definition is related to elements chosen. Figure how the rules and elements are interrelated.
Figure 2. How game elements interact according to the rules
The first action the system. Each given amount of rewarded only oncenumber of times (or have its reward reduced). The player can also choose to join a according to the user one or more questscan comprise tasksuser. Points are used to control the addition, when the player achieves an amount of complete required
An initial set of tasksaccordance to the steps newcomerplace a contributionthe community.following the flow FLOSScoach portal. in the following section
C. Step 3: Designing and Implementing the Gamified Environment
To apply the elements and their ruleof GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issuand wikis. Figure environment. In the fdashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the
Deterding et al. [10], ranking to gamify an environment. In our context,
was to motivate tige among the colleagues.
Step 2: Defining the rules
Rules, when combined, make up a set of provisions that ndition the execution of a game
can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task,
ulating and keeping the players engaged.definition is related to the goal of the game and the game elements chosen. Figure 2 presents a graphical description on how the rules and elements are interrelated.
How game elements interact according to the rules
The first action of a player is the system. Each task, when completed, is rewarded with a given amount of points. To avoid repetition, somerewarded only once and some can only be performed a given number of times (or have its reward reduced). The player can
o join a quest. A given set of according to the user level, and
quests are required to be completed. The tasks, and, when completed, sumare used to control the
tion, when the player achieves an amount of plete required quests, they move to another
An initial set of tasks, questsaccordance to the steps newcomerplace a contribution, learn project practicesthe community. The tasks were created to orient newcomers, following the flow defined FLOSScoach portal. Some examples of tasks
the following section.
Designing and Implementing the Gamified Environment
To apply the elements and their ruleof GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issu
Figure 3 brings an example of thronment. In the figure,
dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the
ranking is a commonly used technique In our context, the goal
to motivate the students, who can gain tige among the colleagues.
Rules, when combined, make up a set of provisions that ndition the execution of a game promoting challenges that
can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task,
ulating and keeping the players engaged.the goal of the game and the game
presents a graphical description on how the rules and elements are interrelated.
How game elements interact according to the rules
of a player is to perform , when completed, is rewarded with a . To avoid repetition, some
and some can only be performed a given number of times (or have its reward reduced). The player can
. A given set of quests, and to advance to a given level
are required to be completed. The , and, when completed, sum
are used to control the rankingtion, when the player achieves an amount of
, they move to another , quests, and levels
accordance to the steps newcomers need to take, learn project practices,
The tasks were created to orient newcomers, by Steinmacher et al.
Some examples of tasks
Designing and Implementing the Gamified
To apply the elements and their rules, we of GitLab and implemented the necessary features. an open source environment that provides Git repository management, code reviews, issue tracking, activity feeds
brings an example of th it is possible to see on the
dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the
is a commonly used technique the goal of using
students, who can gain
Rules, when combined, make up a set of provisions that promoting challenges that
can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task,
ulating and keeping the players engaged. The rules the goal of the game and the game
presents a graphical description on
How game elements interact according to the rules
perform a defined task, when completed, is rewarded with a . To avoid repetition, some actions are
and some can only be performed a given number of times (or have its reward reduced). The player can
quests is unlocked to advance to a given level
are required to be completed. The quests, and, when completed, sum points to the
ranking of users. In tion, when the player achieves an amount of points and
, they move to another level. and levels were defined
to take in order and interact with
The tasks were created to orient newcomers, by Steinmacher et al. [5]
Some examples of tasks are presented
Designing and Implementing the Gamified
, we created a fork of GitLab and implemented the necessary features. GitLab is
rovides Git repository e tracking, activity feeds
brings an example of the gamified ssible to see on the
dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details about the points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests GUI (3), where the user can see what can be done, what is blocked (and the
is a commonly used technique of using
students, who can gain
Rules, when combined, make up a set of provisions that promoting challenges that
can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task,
The rules the goal of the game and the game
presents a graphical description on
task in , when completed, is rewarded with a
actions are and some can only be performed a given
number of times (or have its reward reduced). The player can is unlocked
to advance to a given level quests to the
of users. In and
were defined in in order to
and interact with The tasks were created to orient newcomers,
at are presented
created a fork GitLab is
rovides Git repository e tracking, activity feeds,
e gamified ssible to see on the
dashboard the number of points and the level of a user (1). bout
the points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests
(3), where the user can see what can be done, what is blocked (and the
level requirecan be earned by completinan unblocked quest, a full description of the that need to be accomplished are presented.
For example, for Qthree tasks:
1.
2. 3.
To implement the rules on GitLabalready existing features: issue tracker, thumbs up (similar to the like button on Facebook)task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when players want to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level neliking the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on 4.Figure 3.
Figure 4.
Examples of tasks and the way they were implemented are presented below: Clarifying a doubt with the community
tracand a project owner needs to approve the entry and
level required to open them) and the number of points that can be earned by completinan unblocked quest, a full description of the that need to be accomplished are presented.
Figure 3.
For example, for Qtasks: Post something on the issue tracker (generally an
introduction or a Fork/Clone the repository Setup the local workspace
To implement the rules on GitLabready existing features: issue tracker, thumbs up (similar to
the like button on Facebook)task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when players want to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level ne
ing the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on
Figure 3.
Figure 4. Example of issues usage to implement game rules
xamples of tasks and the way they were implemented are presented below:
Clarifying a doubt with the communitytracker: to earn the points the player places a and a project owner needs to approve the entry and
1
2
3
d to open them) and the number of points that can be earned by completing the task. When a user clicks ian unblocked quest, a full description of the that need to be accomplished are presented.
Quests screen of gamified
For example, for Quest 1, the user
Post something on the issue tracker (generally an troduction or a question)
Fork/Clone the repository Setup the local workspace
To implement the rules on GitLabready existing features: issue tracker, thumbs up (similar to
the like button on Facebook), and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when
to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level needs to
ing the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on
Example of issues usage to implement game rules
xamples of tasks and the way they were implemented
Clarifying a doubt with the community: to earn the points the player places a
and a project owner needs to approve the entry and
Tags reprquest’s task
d to open them) and the number of points that g the task. When a user clicks i
an unblocked quest, a full description of the questthat need to be accomplished are presented.
Quests screen of gamified GitLab
uest 1, the user should accomplish
Post something on the issue tracker (generally an
To implement the rules on GitLab, we made use of ready existing features: issue tracker, thumbs up (similar to
and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when
to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a
to approve it by means of ing the post (using the Thumbs up feature). An example of
the usage of tags and thumbs up is presented on
Example of issues usage to implement game rules
xamples of tasks and the way they were implemented
Clarifying a doubt with the community : to earn the points the player places a
and a project owner needs to approve the entry and
Thumbs up
Tags representing a quest’s task
d to open them) and the number of points that g the task. When a user clicks in
quest and tasks
accomplish
Post something on the issue tracker (generally an
we made use of ready existing features: issue tracker, thumbs up (similar to
and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when
to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a
approve it by means of . An example of
the usage of tags and thumbs up is presented on Figure
Example of issues usage to implement game rules
xamples of tasks and the way they were implemented
via issue : to earn the points the player places a question
and a project owner needs to approve the entry and
Thumbs up
D.
study to assess how the element would motivate and support the students steps towards contributing to an OSS project. Firstly, we chose a projectence manager. JabRef OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.
GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.
Open Source puter Science major atParana. costance of the course, the assignwould contribute some timeronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12 males.
dedicgameceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically ctribute that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 tribution good number, if compared to our previous exper
answer it. The way this approval is made in our enviroment is via thumbs up.
Cloning the repositorydence by posting a vidprove the completion, and a project owner (or a user on a higher level
Setting up a local environmentpository.
Finding a bug to work withthe players can select a bug, and asking the community (via comment) whether they can woAn owner or prove the assign
Reproducing a bugevidence of it in validated by an owner or user at a given level.
Submitting a lated to a reported issue,
Having a pulland merged
D. Step 4: Study conduction
After implementing the rules on GitLabstudy to assess how the element would motivate and support the students overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we chose a projectence manager. JabRef OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.
JabRef project was imported to a local private instance of GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.
The study was conducted with students attending to the Open Source Developmentputer Science major atParana. In this course, students generally are requested to contribute to an OSS project as an stance of the course, the assignwould contribute some time. Out of 18 students, 17 accepted to ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12 males.
Before starting their assignment, dedicated to explaining the environment, the rules of the game, and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically ctribute with code to that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 tribution by the end of the asgood number, if compared to our previous exper
answer it. The way this approval is made in our enviroment is via thumbs up. Cloning the repository: the players need to provide evdence by posting a video or a set of screenshots that prove the completion, and a project owner (or a user on a
level) needs to approve itSetting up a local environment
Finding a bug to work withplayers can select a bug, and asking the community
(via comment) whether they can woor a user on a higher level
prove the assignment, and often provide support.Reproducing a bug: reproduce a bug
of it in a comment validated by an owner or user at a given level.Submitting a pull-requestlated to a reported issue, independently of its ac
pull-request acceptedand merged, the player earns points.
Study conduction
After implementing the rules on GitLabstudy to assess how the element would motivate and support
overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we chose a project: JabRef2, an open source bibliography ence manager. JabRef is written in Java and is a consolidated OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.
JabRef project was imported to a local private instance of GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.
The study was conducted with students attending to the Development course, in the last year of Co
puter Science major at Federal University of Technology In this course, students generally are requested to
ntribute to an OSS project as an stance of the course, the assignwould contribute using the gamified environmen
. Out of 18 students, 17 accepted to ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12
Before starting their assignment, explaining the environment, the rules of the
and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically c
with code to JabRef inthat the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9
by the end of the asgood number, if compared to our previous exper
2 http://www.jabref.org
answer it. The way this approval is made in our enviro
: the players need to provide eveo or a set of screenshots that
prove the completion, and a project owner (or a user on a ) needs to approve it using the thumbs up
Setting up a local environment: similar to clone the r
Finding a bug to work with: by following somplayers can select a bug, and asking the community
(via comment) whether they can work on the bug or not. a user on a higher level can answer and a
ment, and often provide support.: reproduce a bug
a comment to the bug, which needs to be validated by an owner or user at a given level.
request: submitting a pullindependently of its acaccepted: after being accepted
, the player earns points.
After implementing the rules on GitLabstudy to assess how the element would motivate and support
overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we
an open source bibliography written in Java and is a consolidated
OSS project. In addition, one of the authorsproject maintainers, which facilitated instrumenting it.
JabRef project was imported to a local private instance of GitLab, including all its versioning history and issues. The most recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.
The study was conducted with students attending to the course, in the last year of Co
Federal University of Technology In this course, students generally are requested to
ntribute to an OSS project as an assignment. In this istance of the course, the assignment was changed
using the gamified environmen. Out of 18 students, 17 accepted to
ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12
Before starting their assignment, a twoexplaining the environment, the rules of the
and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically consisted of attempting to co
JabRef in 20 days. It was not expected that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9
by the end of the assignment (53%), good number, if compared to our previous exper
http://www.jabref.org
answer it. The way this approval is made in our enviro
: the players need to provide eveo or a set of screenshots that
prove the completion, and a project owner (or a user on a using the thumbs up. : similar to clone the r
following some hints, players can select a bug, and asking the community
rk on the bug or not. can answer and a
ment, and often provide support. : reproduce a bug locally, provide
to the bug, which needs to be validated by an owner or user at a given level.
submitting a pull-request rindependently of its acceptance
: after being accepted
After implementing the rules on GitLab, we designed a study to assess how the element would motivate and support
overcoming orientation barriers on their first steps towards contributing to an OSS project. Firstly, we
an open source bibliography refewritten in Java and is a consolidated
authors is one of the project maintainers, which facilitated instrumenting it.
JabRef project was imported to a local private instance of ry and issues. The
most recent issues were tagged according to their level of
The study was conducted with students attending to the course, in the last year of Co
Federal University of Technology In this course, students generally are requested to
assignment. In this iwas changed so they
using the gamified environment during . Out of 18 students, 17 accepted to use the env
ronment and consented to be part of the study. Students’ ages varied from 20 to 25 years old, including 5 females and 12
a two-hour class was explaining the environment, the rules of the
and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handed-over the
attempting to coIt was not expected
that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 placed a co
signment (53%), which is a good number, if compared to our previous experiences [5].
answer it. The way this approval is made in our environ-
: the players need to provide evi-eo or a set of screenshots that
prove the completion, and a project owner (or a user on a
: similar to clone the re-
e hints, players can select a bug, and asking the community
rk on the bug or not. can answer and ap-
locally, provide to the bug, which needs to be
request re-ceptance.
: after being accepted
we designed a study to assess how the element would motivate and support
on their first steps towards contributing to an OSS project. Firstly, we
refer-written in Java and is a consolidated
is one of the
JabRef project was imported to a local private instance of ry and issues. The
most recent issues were tagged according to their level of
The study was conducted with students attending to the course, in the last year of Com-
Federal University of Technology – In this course, students generally are requested to
assignment. In this in-so they during
use the envi-Students’ ages
varied from 20 to 25 years old, including 5 females and 12
hour class was explaining the environment, the rules of the
and presenting JabRef. In addition, the students re-ceived a document presenting the rules and some guidance
over the attempting to con-
It was not expected that the students succeeded by the end of the 20 days period.
placed a con-which is a
.
At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5 points Likertenable students to share their feedback.tionnaire was to assess to what extent gamified overcome session was conducted to help us
In this sectionmostly administered after the assignment period. The questionnaire was organized of Likertmotivated and supported newcomers’ first steps; one section with fourgrade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess theTo facilitate the understanding, wing the questionnaire structure.
A. Quest
The step process to guide newcomers, keeping them motivated and oriented. shares dents answered these three 5pletely
Q1. Q2.
Q3.
Q4.
As it can be seen
were viewed by the studentsstudents evaluated tively.
In addition, orient the contribution. as a guide, 9 people became neutral or disagreed.
At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5
Likert-scale items, and some openenable students to share their feedback.tionnaire was to assess to what extent gamified environment motivated and helped the students to overcome orientation barriers.session was conducted to help us
In this section, we bring the results mostly on the answers we obtained from the questionnaire administered after the assignment period. The questionnaire was organized in the following way: one section composed of Likert-scale items motivated and supported newcomers’ first steps; one section
four closed-ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess the
facilitate the understanding, wing the questionnaire structure.
Quests and Levels
The quest element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated and oriented. The level
the goal of guiding newcomers. dents answered these three 5
Disagree to Completely Q1. The quests stepsQ2. Quests were useful to orient me
contribution;Q3. The quest element kept me motivated to perform the
proposed tasksQ4. The possibility of
me to change levelAs it can be seen
viewed by the studentsstudents evaluated Q3
Figure 5. Results for
In addition, most part of the newcomers found it useful to orient the contribution. as a guide, 9 people became neutral or disagreed.
At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5
scale items, and some openenable students to share their feedback.tionnaire was to assess to what extent
environment motivated and helped the students to orientation barriers. A half
session was conducted to help us to
IV. RESULTS
we bring the results on the answers we obtained from the questionnaire
administered after the assignment period. The questionnaire the following way: one section composed
scale items to assess how eachmotivated and supported newcomers’ first steps; one section
ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess the future use of a
facilitate the understanding, we present the ring the questionnaire structure.
s and Levels
element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated
level element is linked to the quests, and the goal of guiding newcomers.
dents answered these three 5-points to Completely Agree) items:
steps guided my contribution process;Quests were useful to orient me
; The quest element kept me motivated to perform the proposed tasks and to contributeThe possibility of unblockingme to change levels.
As it can be seen in Figure 5, Quests viewed by the students as good motivators, since
Q3 positively and 14 evalu
Results for Quests-
most part of the newcomers found it useful to orient the contribution. However, regardingas a guide, 9 people became neutral or disagreed.
At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5
scale items, and some open-ended question to enable students to share their feedback. The goal of tionnaire was to assess to what extent each element
environment motivated and helped the students to A half-hour, informal debrief
to understand some results.
ESULTS
we bring the results of our study, relying on the answers we obtained from the questionnaire
administered after the assignment period. The questionnaire the following way: one section composed
to assess how each gamesmotivated and supported newcomers’ first steps; one section
ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four L
future use of a gamified environment.e present the results reflec
element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated
element is linked to the quests, and the goal of guiding newcomers. Therefore, the st
points Likert-scale (from CoAgree) items:
guided my contribution process;Quests were useful to orient me to place
The quest element kept me motivated to perform the and to contribute;
ing new quests
, Quests allied with Levels as good motivators, since
and 14 evaluated Q4 pos
-related questions
most part of the newcomers found it useful to regarding the use of quests
as a guide, 9 people became neutral or disagreed.
At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5
ended question to The goal of the ques-each element and the
environment motivated and helped the students to hour, informal debrief
understand some results.
of our study, relying on the answers we obtained from the questionnaire
administered after the assignment period. The questionnaire the following way: one section composed
games element motivated and supported newcomers’ first steps; one section
ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Likert-scale
gamified environment. esults reflect-
element was chosen to help creating a step-by-step process to guide newcomers, keeping them motivated
element is linked to the quests, and Therefore, the stu-
from Com-
guided my contribution process; to place my
The quest element kept me motivated to perform the
new quests motivated
allied with Levels as good motivators, since 12
ated Q4 posi-
most part of the newcomers found it useful to the use of quests
as a guide, 9 people became neutral or disagreed. During an
informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that needtasks within a quest
are good to motivate ered good newcomers, however, per
B.
studentsfore, the following items were administered:
Q2 are similar. They snegative answers. motivatorthan 30% disagreed with at least one sentence related to motivationthat For example, the profile of the players, the public exposition, or the absence of a reward related to the points.
we see noproviding feedback on the tasks performed by the students.
providing some players, but not others.
C.
tivate the students. assess the element:
informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that needs to be trodden, but, the obligation to fulfill all specific tasks within a quest
In summary,are good to motivate ered good to show the newcomers, however, perform the steps would be beneficial.
B. Points
The main goal students and give them feedbackfore, the following items were administered:
Q1. Points element motivated me to keep contributingQ2. The more points I got
earn, so I could increase my Level and my Ranking.Q3. Points
tasks I performedAs presented by Figure
Q2 are similar. They snegative answers. motivator for approximately half of the students, but more than 30% disagreed with at least one sentence related to motivation. During the debrief session, it wathat there are other factors that influence For example, the profile of the players, the public exposition, or the absence of a reward related to the points.
On the other hand, when we observe the results for Q3, we see no disagreement, indicating that Points were good on providing feedback on the tasks performed by the students.
Figure 6.
Therefore, oproviding feedback some players, but not others.
C. Ranking
Allied with points, the main goal of Ranking was to mtivate the students. assess the element:
Q1. I wanted could improve my
Q2. The Rankipare my performance with my colleagues
informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that
to be trodden, but, the obligation to fulfill all specific tasks within a quest bothered some participants
In summary, our results indicate thatare good to motivate the students.
to show the steps that need tnewcomers, however, adding more flexibility on how to
form the steps would be beneficial.
The main goal of choosingand give them feedback
fore, the following items were administered:Points element motivated me to keep contributingThe more points I gotearn, so I could increase my Level and my Ranking.Points were useful to provide me tasks I performed.
As presented by Figure 6Q2 are similar. They show a balance between positive and negative answers. The points themselves were
for approximately half of the students, but more than 30% disagreed with at least one sentence related to
During the debrief session, it waother factors that influence
For example, the profile of the players, the public exposition, or the absence of a reward related to the points.
On the other hand, when we observe the results for Q3, disagreement, indicating that Points were good on
providing feedback on the tasks performed by the students.
Figure 6. Results for Points
Therefore, our results indicate thatfeedback to the newcomers
some players, but not others.
Allied with points, the main goal of Ranking was to mtivate the students. In this case, only assess the element:
anted to perform more tasks and quests so I could improve my RankingThe Ranking motivated me because I could pare my performance with my colleagues
informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that
to be trodden, but, the obligation to fulfill all specific bothered some participants
our results indicate that queststhe students. Quests were also consi
that need to be followed by the adding more flexibility on how to
form the steps would be beneficial.
choosing points was to motivate the and give them feedback about their actions
fore, the following items were administered:Points element motivated me to keep contributingThe more points I got, the more I was motivated to earn, so I could increase my Level and my Ranking.
were useful to provide me
6, the answers to items Q1 and how a balance between positive and
he points themselves were for approximately half of the students, but more
than 30% disagreed with at least one sentence related to During the debrief session, it wa
other factors that influence students’ answers. For example, the profile of the players, the public exposition, or the absence of a reward related to the points.
On the other hand, when we observe the results for Q3, disagreement, indicating that Points were good on
providing feedback on the tasks performed by the students.
Results for Points-related questions
ur results indicate that pointsthe newcomers, and can moti
Allied with points, the main goal of Ranking was to mIn this case, only two items were used to
perform more tasks and quests so I Ranking.
ng motivated me because I could pare my performance with my colleagues
informal debrief session after the study, we found that apossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that
to be trodden, but, the obligation to fulfill all specific bothered some participants.
quests and levelswere also consi
o be followed by the adding more flexibility on how to
was to motivate the about their actions. Ther
fore, the following items were administered: Points element motivated me to keep contributing
the more I was motivated to earn, so I could increase my Level and my Ranking.
were useful to provide me feedback on the
, the answers to items Q1 and how a balance between positive and
he points themselves were considered for approximately half of the students, but more
than 30% disagreed with at least one sentence related to During the debrief session, it was possible to feel
students’ answers. For example, the profile of the players, the public exposition, or the absence of a reward related to the points.
On the other hand, when we observe the results for Q3, disagreement, indicating that Points were good on
providing feedback on the tasks performed by the students.
related questions
points are good on , and can motivate
Allied with points, the main goal of Ranking was to mitems were used to
perform more tasks and quests so I
ng motivated me because I could copare my performance with my colleagues.
t a possible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that
to be trodden, but, the obligation to fulfill all specific
levels were also consid-
o be followed by the adding more flexibility on how to
was to motivate the There-
Points element motivated me to keep contributing. the more I was motivated to
earn, so I could increase my Level and my Ranking. feedback on the
, the answers to items Q1 and how a balance between positive and
considered for approximately half of the students, but more
than 30% disagreed with at least one sentence related to to feel
students’ answers. For example, the profile of the players, the public exposition,
On the other hand, when we observe the results for Q3, disagreement, indicating that Points were good on
providing feedback on the tasks performed by the students.
are good on vate
Allied with points, the main goal of Ranking was to mo-items were used to
perform more tasks and quests so I
com-
As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagrQ2). The results aretained for the points, which makes sense, since points and ranking are can be other factors that influenced these results. So, astudy analyzing the profile, goalsdents along with the gamified environment, would provide us explanation on these results.
Our results indicated that motivate
D. Comparing the elements
On the second part of the questionnaire, the students were asked to grade the elements orientation the elements providetunity to choose which elements motivated them and useful on orienting the contribution process.
Figure by the students when we asked related to newcomersmuch havebarriers?”
As it is possible to observe, and in consonance with the results presented before, the on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9
Figure 8.
Pointsstudents
As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagrQ2). The results are, somehow,tained for the points, which makes sense, since points and ranking are expected to be can be other factors that influenced these results. So, astudy analyzing the profile, goalsdents along with the gamified environment, would provide us explanation on these results.
Figure 7. Results for Ranking
Our results indicated that motivate some players, but not others.
Comparing the elements
On the second part of the questionnaire, the students were asked to grade the elements orientation the elements providetunity to choose which elements motivated them and useful on orienting the contribution process.
Figure 8 presents the distribution of the grades provided by the students when we asked related to newcomers
have each element helpbarriers?”
As it is possible to observe, and in consonance with the results presented before, the on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9
Figure 8. On a 10-points scale hmitigate/reduce these barriers
Points were also well graded. More than a half of the students evaluated with 7 or a better grade (including two
As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagr
, somehow, aligned with the results otained for the points, which makes sense, since points and
expected to be complementary. Once again, there can be other factors that influenced these results. So, astudy analyzing the profile, goals, dents along with the gamified environment, would provide us explanation on these results.
Results for Ranking
Our results indicated that Rankingsome players, but not others.
Comparing the elements
On the second part of the questionnaire, the students were asked to grade the elements orientation the elements provide. They also had the oppotunity to choose which elements motivated them and useful on orienting the contribution process.
presents the distribution of the grades provided by the students when we asked “Regarding the barriers related to newcomers’ orientation, in a 10
each element helped
As it is possible to observe, and in consonance with the results presented before, the quests on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9, and one gave a 10.
points scale how much have each element helped tomitigate/reduce these barriers
were also well graded. More than a half of the evaluated with 7 or a better grade (including two
As it is possible to notice in Figure 7, just over half of the students (9) agreed that the Ranking element was a motivtional factor, while few others disagreed (4 on Q1 and 3 on
aligned with the results otained for the points, which makes sense, since points and
complementary. Once again, there can be other factors that influenced these results. So, a
and attitudes of the stdents along with the gamified environment, would provide
Results for Ranking-related questions
Ranking, just like Pointssome players, but not others.
On the second part of the questionnaire, the students were asked to grade the elements according to how much
They also had the oppotunity to choose which elements motivated them and useful on orienting the contribution process.
presents the distribution of the grades provided “Regarding the barriers
orientation, in a 10-point scale, how to mitigate/reduce these
As it is possible to observe, and in consonance with the played an important role
on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated
and one gave a 10.
ow much have each element helped tomitigate/reduce these barriers
were also well graded. More than a half of the evaluated with 7 or a better grade (including two
just over half of the students (9) agreed that the Ranking element was a motiva-
eed (4 on Q1 and 3 on aligned with the results ob-
tained for the points, which makes sense, since points and complementary. Once again, there
can be other factors that influenced these results. So, another and attitudes of the stu-
dents along with the gamified environment, would provide
related questions
Points, can
On the second part of the questionnaire, the students according to how much
They also had the oppor-tunity to choose which elements motivated them and were
presents the distribution of the grades provided “Regarding the barriers
point scale, how to mitigate/reduce these
As it is possible to observe, and in consonance with the played an important role
on orienting the students towards their contribution. By look-ing at the raw data, we can see that seven people evaluated
and one gave a 10.
ow much have each element helped to
were also well graded. More than a half of the evaluated with 7 or a better grade (including two
10). The possible expining feedback to students.
part of the grades equal or below 5. These results for the Rankingelement was not orienting, but motivating the players. Hoever, we expected a better evaluation for sincethe debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.
the students to tell which elements were motivators and useful for orienting them. The results are presented in TaIwith previous results. role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking elemstudents.
not considervate studentsthink of works in conjunction with
Which element(s) were useful to
orient
Which element(s)
keep contributing?
Which element you
anymore?
E.
evaluate the experience of the students with a gamified OSS environment.provided with the following sentences:
agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 environment supported them during their contribution prcess.
10). The possible expin Section IV.Bing feedback to students.
Levels and part of the grades equal or below 5. These results for the Ranking element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for since it plays an orientation role along with the the debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.
As aforementioned, to crossthe students to tell which elements were motivators and useful for orienting them. The results are presented in TaI. It is possible to notice that the results are, partially, aligned with previous results. role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking element was considered as a good motivator by 47% of the students.
It is also possible to observe in Table not considered vate students. Once again, this can be related think of levels, as an independent element or something that works in conjunction with
TABLE I.
Which element(s) were useful to
orient your contribution?
Which element(s) motivated
keep contributing?
Which element you
anymore?
E. Evaluating the environment
The last section of the evaluate the experience of the students with a gamified OSS environment. The students answered two Likertprovided with the following sentences:
Q1. I would use the gamified GitLab in the futureQ2. The gamified
contribution process.
As it is possible to observe on Figure agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 environment supported them during their contribution prcess.
10). The possible explanation relates to the results presented IV.B, which shows that points are good on provi
ing feedback to students. and Ranking had worse results,
part of the grades equal or below 5. These results for the element are not surprising, since the goal of this
element was not orienting, but motivating the players. Hoever, we expected a better evaluation for
it plays an orientation role along with the the debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.
mentioned, to crossthe students to tell which elements were motivators and useful for orienting them. The results are presented in Ta
It is possible to notice that the results are, partially, aligned with previous results. Points and role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking
ent was considered as a good motivator by 47% of the
It is also possible to observe in Table important to provide
. Once again, this can be related , as an independent element or something that
works in conjunction with quests
IMPORTANCE OF ELEMENT
ORIENTATION AND MOTI
Quests
Which element(s) were useful to
your contribution?
10
(59%)
motivated you to 10
(59%)
Which element you would not use
(6%)
Evaluating the environment
section of the questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS
The students answered two Likertprovided with the following sentences:
I would use the gamified GitLab in the futureThe gamified GitLab supported me dcontribution process.
As it is possible to observe on Figure agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 students (82%)environment supported them during their contribution pr
lanation relates to the results presented , which shows that points are good on provi
had worse results, part of the grades equal or below 5. These results for the
element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for
it plays an orientation role along with the the debrief, we understand that a possible explanation is that the students evaluated the level of information offered witout considering it as a part of the flow that guided them from
mentioned, to cross-check our results, we asked the students to tell which elements were motivators and useful for orienting them. The results are presented in Ta
It is possible to notice that the results are, partially, aligned and Quests played an
role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking
ent was considered as a good motivator by 47% of the
It is also possible to observe in Table I important to provide orientation or
. Once again, this can be related , as an independent element or something that
quests.
MPORTANCE OF ELEMENTS WITH REGARD TO
ORIENTATION AND MOTIVATION
Quests Levels Points
10
(59%)
4
(24%) (53%)
10
(59%)
2
(12%)
10
(59%)
1
(6%)
5
(29%) (18%)
Evaluating the environment
questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS
The students answered two Likertprovided with the following sentences:
I would use the gamified GitLab in the futureGitLab supported me d
contribution process.
As it is possible to observe on Figure 9, agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More
(82%) pointed that the gamified environment supported them during their contribution pr
lanation relates to the results presented , which shows that points are good on provi
presenting most part of the grades equal or below 5. These results for the
element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for Levels element,
it plays an orientation role along with the Quests. From possible explanation is that
information offered witout considering it as a part of the flow that guided them from
check our results, we asked the students to tell which elements were motivators and useful for orienting them. The results are presented in Table
It is possible to notice that the results are, partially, aligned played an important
role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking
ent was considered as a good motivator by 47% of the
that Levels were orientation or to mot
. Once again, this can be related to how students , as an independent element or something that
S WITH REGARD TO
Points Ranking None
9
(53%)
3
(18%)
10
(59%)
8
(47%)
3
(18%)
7
(41%) (29%)
questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS
The students answered two Likert-scale items,
I would use the gamified GitLab in the future. GitLab supported me during the
8 students (53%) agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More
pointed that the gamified environment supported them during their contribution pr
lanation relates to the results presented , which shows that points are good on provid-
presenting most part of the grades equal or below 5. These results for the
element are not surprising, since the goal of this element was not orienting, but motivating the players. How-
element, From
possible explanation is that information offered with-
out considering it as a part of the flow that guided them from
check our results, we asked the students to tell which elements were motivators and
ble It is possible to notice that the results are, partially, aligned
important role on orientation, and, to some extent, motivated the stu-dents during their contributions. We also see that Ranking
ent was considered as a good motivator by 47% of the
were moti-
how students , as an independent element or something that
None
-
-
5
(29%)
questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS
scale items,
uring the
8 students (53%) agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More
pointed that the gamified environment supported them during their contribution pro-
Figure 9.
In this papergamified environment dents toa subset of gaming elementsmotivate students to place a first contribution to an OSS project
By conducting thisthat using were effective on barriers andof the questtion. The use of provide feedback to the students, and keeping them motivaed to contribute.
In addition, we could observe thatsome extent, Ranking, keep contributing to OSS projeno external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.
Unlike the other elementsLevel did not sthem overcompointed that the possibility of ed them to change level
The results presented here, although preliminary, can ptentially benegame design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi
Although the results indicate that a gamified environment can support can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivatin an academic environment, to put them in touchcodebase and issuesOSS prkeeping a coFurther and longtigate it, since it would be necessary to change internal prlems.
In the future, in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and retention of students.
Figure 9. Results for the evaluation of the gamified GitLab as a and a way to orient the students
V.
In this paper, we presented a preliminary study on how a gamified environment dents to contribute to a subset of gaming elementsmotivate students to
place a first contribution to an OSS projectBy conducting this
that using quests composed of objective and sequential tasks, effective on mitigating/reducing t
barriers and supporting newcomers’ quests provided a step
The use of pointsprovide feedback to the students, and keeping them motivaed to contribute.
In addition, we could observe thatsome extent, Ranking, keep contributing to OSS projeno external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.
Unlike the other elementsdid not seem to
them overcoming orientation barriers. However, students pointed that the possibility of ed them to change level
The results presented here, although preliminary, can ptentially benefit OSS communities, who can make use of game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi
Although the results indicate that a gamified environment can support newcomerscan foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat
academic environment, to put them in touchodebase and issues, and to contribute for the first time to an
OSS project. This process is different from the process of keeping a community active or of retaining contributors. Further and long-term research wtigate it, since it would be necessary to change internal processes, which can lead to other types of pro
In the future, we plan to further evaluate these elements in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and retention of students.
Results for the evaluation of the gamified GitLab as a and a way to orient the students
V. CONCLUSION
we presented a preliminary study on how a gamified environment can help orienting and
OSS projects. We specifically analyzed a subset of gaming elements and assessed their ability to
to overcome orientation barriers inherent place a first contribution to an OSS projectBy conducting this study with students, we could observe
composed of objective and sequential tasks, mitigating/reducing t
supporting newcomers’ provided a step-by-step towards the contrib
points was also viewedprovide feedback to the students, and keeping them motiva
In addition, we could observe thatsome extent, Ranking, can be used to motivate students to keep contributing to OSS projects. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.
Unlike the other elements, according to the studentseem to keep students
orientation barriers. However, students pointed that the possibility of unlockinged them to change levels.
The results presented here, although preliminary, can pfit OSS communities, who can make use of
game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi
Although the results indicate that a gamified environment newcomers, we are not able to say whether this
can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat
academic environment, to put them in touch, and to contribute for the first time to an
ject. This process is different from the process of munity active or of retaining contributors.
term research would be necessary to invetigate it, since it would be necessary to change
cesses, which can lead to other types of pro
we plan to further evaluate these elements in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to host projects conducted by students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and
Results for the evaluation of the gamified GitLab as a and a way to orient the students
ONCLUSION
we presented a preliminary study on how a orienting and engaging st
OSS projects. We specifically analyzed and assessed their ability to orientation barriers inherent
place a first contribution to an OSS project. study with students, we could observe
composed of objective and sequential tasks, mitigating/reducing the orientation related
supporting newcomers’ first steps. The structure step towards the contrib
viewed as a good strategy to provide feedback to the students, and keeping them motiva
In addition, we could observe that Quests, Pointscan be used to motivate students to
cts. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students
, according to the studentskeep students motivated, or
orientation barriers. However, students unlocking new quests
The results presented here, although preliminary, can pfit OSS communities, who can make use of
game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during this journey.
Although the results indicate that a gamified environment are not able to say whether this
can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat
academic environment, to put them in touch , and to contribute for the first time to an
ject. This process is different from the process of munity active or of retaining contributors.
ould be necessary to invetigate it, since it would be necessary to change communities’
cesses, which can lead to other types of pro
we plan to further evaluate these elements in a longer study, focusing on analyzing behavior and asessing the use of external rewards. It is part of this project
host projects conducted by students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and
Results for the evaluation of the gamified GitLab as a motivator
we presented a preliminary study on how a engaging stu-
OSS projects. We specifically analyzed and assessed their ability to orientation barriers inherent
study with students, we could observe composed of objective and sequential tasks,
he orientation related . The structure
step towards the contribu-as a good strategy to
provide feedback to the students, and keeping them motivat-
Quests, Points, and, to can be used to motivate students to
cts. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students
, according to the students, motivated, or to help
orientation barriers. However, students quests motivat-
The results presented here, although preliminary, can po-fit OSS communities, who can make use of
game design elements to support newcomers and help engag-ing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribu-
Although the results indicate that a gamified environment
are not able to say whether this can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivate students
with a real , and to contribute for the first time to an
ject. This process is different from the process of munity active or of retaining contributors.
ould be necessary to inves-communities’
cesses, which can lead to other types of prob-
we plan to further evaluate these elements behavior and as-
sessing the use of external rewards. It is part of this project to host projects conducted by
students and faculties of the university and create a program to reward top contributors and to validate contributions as credits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and
REFERENCES
[1] V. Goduguluri, T. Kilamo, and I. Hammouda, “Kommgame: A reputation environment for teaching open source software,” in IFIP International Conference on Open Source Systems, 2011, pp. 312–315.
[2] B. Morgan and C. Jensen, “Lessons Learned from Teaching Open Source Software Development,” in 10th International Conference on Open Source Systems, OSS 2014, San José, Costa Rica, May 6-9, 2014., 2014, pp. 133–142.
[3] D. M. Nascimento, K. Cox, T. Almeida, W. Sampaio, R. A. Bittencourt, R. Souza, and C. Chavez, “Using Open Source Projects in software engineering education: A systematic mapping study,” in 2013 IEEE Frontiers in Education Conference, 2013, pp. 1837–1843.
[4] I. Steinmacher, T. Conte, M. A. Gerosa, and D. Redmiles, “Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects,” in Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 2015, pp. 1379–1392.
[5] I. Steinmacher, T. Conte, C. Treude, and M. A. Gerosa, “Overcoming Open Source Project Entry Barriers with a Portal for Newcomers,” in 38th International Conference on Software Engineering, 2016 (ICSE 2016), 2016.
[6] M. Zhou and A. Mockus, “Does the initial environment impact the future of developers,” in Proceedings of the 33rd International Conference on Software Engineering, 2011, pp. 271–280.
[7] G. Canfora, M. di Penta, R. Oliveto, and S. Panichella, “Who is Going to Mentor Newcomers in Open Source Projects?,” in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, 2012, pp. 44:1–44:11.
[8] A. Schilling, S. Laumer, and T. Weitzel, “Who Will Remain? An Evaluation of Actual Person-Job and Person-Team Fit to Predict Developer Retention in FLOSS Projects,” in Proceedings of the 2012 45th Hawaii International Conference on System Sciences, 2012, pp. 3446–3455.
[9] M. Zhou and A. Mockus, “What make long term contributors: Willingness and opportunity in OSS community,” in Proceedings of the 34th International Conference on Software Engineering, 2012, pp. 518–528.
[10] Deterding, Sebastian, Dixon, Dan, Khaled, Rilla, Nacke, and Lennart, “From Game Design Elements to Gamefulness: Defining "Gamification,” in Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 2011, pp. 9–15.
[11] A. P. O. Bertholdo and M. A. Gerosa, “Promoting Engagement in Open Collaboration Communities by Means of Gamification,” in HCI International 2016 – Posters’ Extended Abstracts: 18th International Conference, HCI International 2016 Toronto, Canada, July 17–22, 2016 Proceedings, Part II, C. Stephanidis, Ed. Cham: Springer International Publishing, 2016, pp. 15–20.
[12] S. K. Sheth, J. S. Bell, and G. E. Kaiser, “Increasing Student Engagement in Software Engineering with Gamification,” 2012.
[13] R. van der V. Philipp Lombriser, “Improving the Quality of the Software Development Lifecycle with Gamification,” p. 22, 2014.
[14] O. Pedreira, F. García, N. Brisaboa, and M. Piattini, “Gamification in software engineering – A systematic mapping,” Information and Software Technology, vol. 57, pp. 157–168, 2015.
[15] A. Bartel and G. Hagel, “Gamifying the learning of design patterns in software engineering education,” in Global Engineering Education Conference (EDUCON), 2016 IEEE, 2016, pp. 74–79.
[16] D. Dicheva, C. Dichev, G. Agre, and G. Angelova, “Gamification in Education: A Systematic Mapping Study.,” Educational Technology & Society, vol. 18, no. 3, pp. 75–88, 2015.
[17] F. Steffens, S. Marczak, F. F. Filho, C. Treude, L. Singer, D. Redmiles, and B. Al-Ani, “Using Gamification as a Collaboration Motivator for Software Development Teams: A Preliminary Framework,” in SBSC ’15: Brazilian Symp. in Collaborative Systems, 2015, p. 10pp.
[18] A. Marczewski, Gamification: a simple introduction. Andrzej Marczewski, 2013, p. 153.
[19] G. Surendeleg, V. Murwa, H.-K. Yun, and Y. S. Kim, “The role of gamification in education a literature review,” Contemporary Engineering Sciences, vol. 7, no. 2932, pp. 1609–1616, 2014.
[20] D. J. Dubois and G. Tamburrelli, “Understanding Gamification Mechanisms for Software Development,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, 2013, pp. 659–662.
[21] A. de Melo, M. Hinz, G. Scheibel, C. Diacui Medeiros Berkenbrock, I. Gasparini, and F. Baldo, “Version Control System Gamification: A Proposal to Encourage the Engagement of Developers to Collaborate in Software Projects,” in Social Computing and Social Media, Springer, 2014, pp. 550–558.
[22] R. Lotufo, L. Passos, and K. Czarnecki, “Towards improving bug tracking systems with game mechanisms,” in Proceedings of the 9th IEEE Working Conference on Mining Software Repositories, 2012, pp. 2–11.
[23] W. Snipes, A. R. Nair, and E. Murphy-Hill, “Experiences Gamifying Developer Adoption of Practices and Tools,” in Companion Proceedings of the 36th International Conference on Software Engineering, 2014.
[24] D. F. Bacon, Y. Chen, D. Parkes, and M. Rao, “A Market-based Approach to Software Evolution,” in Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications, 2009, pp. 973–980.
[25] S. Sheth, J. Bell, and G. Kaiser, “Halo (highly addictive, socially optimized) software engineering,” in Proceedings of the 1st International Workshop on Games and Software Engineering, 2011, pp. 29–32.
[26] C. Thomas and K. Berkling, “Redesign of a gamified software engineering course,” in Interactive Collaborative Learning (ICL), 2013 International Conference on, 2013, pp. 778–786.
[27] Jan Nonnen Christian R. Prause and M. Vinkovits, “A Field Experiment on Gamication of Code Quality in Agile Development,” 2012.
[28] J. M. Rojas and G. Fraser, “Teaching Mutation Testing using Gamification,” in European Conference on Software Engineering Education (ECSEE), 2016.
[29] I. Steinmacher, T. Conte, M. A. Gerosa, and D. F. Redmiles, “Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects,” in Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 2015, pp. 1–13.
[30] M. Csikszentmihaly, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play, 25th ed. Hoboken, NJ, EUA: Jossey-Bass, 2000, p. 231.