Date post: | 05-Dec-2014 |
Category: |
Technology |
Upload: | ptidej-team |
View: | 184 times |
Download: | 0 times |
Who is going to Mentor Newcomers in Open Source Projects?
Gerardo Massimiliano Rocco Sebastiano Canfora Di Penta Oliveto Panichella
Context and Motivations • Software Development
How? • Training via Mentoring
Case Study •Explorative analysis
•Recommendation system evaluation
Training Project Newcomers
With a GOOD TRAINING
Can immediately start to work
ACTIVELY
Newcomer
Zhou and Mockus
Better training from
Senior Developers
Newcomer
Previous Work...
Low Sociability
Previous Work...
Dagenais et al.
MENTOR
Newcomer
Mentoring of project newcomers is highly desirable
Characteristics of a good Mentor
Sources of Information
SVN GIT CVS
Small Projects: find Mentors
is a trivial problem
Large Projects: : find Mentors
is not a trivial problem
Mentoring Small/large Projects
.........
YODA (Young and newcOmer Developer Assistant)
Approach for Mentors Identification
in Open Source Projects
SVN GIT CVS
YODA: two pashes
?
What factor can be used to identify
mentors?
What factor can be used to identify mentors?
RQ1: Identify past mentors
How does Arnetminer works?
f1: they published Many papers Together f2: advisor published More than the Student f3: advisor older than the student f4: student published her first paper(s) with the advisor
Ranks pairs of researchers according to four factors:
Time
F1: Exchanged emails
Heuristics to identify Mentors
When Alice joins the project
F1: Exchanged emails
Heuristics to identify Mentors
Time
F2: overall amount of emails
Heuristics to identify Mentors
F2: overall amount of emails
Heuristics to identify Mentors
F2: overall amount of emails
Heuristics to identify Mentors
F3: project age Heuristics to identify Mentors
Time
F3: project age Heuristics to identify Mentors
F4: newcomer early emails
Heuristics to identify Mentors
Time
First emails by Alice in the project
F4: newcomer early emails
Heuristics to identify Mentors
F5: Commits
Heuristics to identify Mentors
When Alice joins the project
Time
F5: Commits
Heuristics to identify Mentors
What factors can be used to identify mentors?
Aggregating the factors
5
1i
ii fw
Recommend Mentors
Time
Recommend Mentors
Time
Recommend Mentors
Time t
Recommend Mentors
Time t
Mentor with adequate skills
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
t
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
t
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
t
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
t
Recommend Mentors
Time
Inspired to the work On Bug Triaging by J. Anvik et al.
2011
t
DICE SIMILARITY
Empirical Study
• Goal: analyze data from mailing lists and versioning
systems
• Purpose: investigating which factors can be used to
identify mentors
• Quality focus: recommend mentors in software
projects
• Context: mailing lists and versioning systems of five software
Apache, FreeBSD, PostgreSQL, Python and Samba
Apache FreeBSD PostgreSQL Python Samba
Period
(Training set) 08/2001-03/2002 11/1998-02/2000 10/1998-05/2001 05/2000-05/2001 04/1998-09/2000
Period
(Test set) 04/2002-12/2008 03/2000-10/2008 06/2001-03/2008 06/2001-12/2008 10/2000-12/2008
# of Mentors
(Training set) 19 65 10 28 17
# of
Newcomers
(Training set) 13 33 8 32 33
# of
Newcomers
(Test set) 13 33 7 31 33
Context
Training and Test sets for evaluating Yoda.
Research Questions
?
RQ1: How can we identify mentors from the past history of a software project? SCORE
2.5
1.5
1.5
1.5
1.5
1.5
……….
COUPLES
……….
5
1i
ii fw
RQ1: How can we identify mentors from the past history of a software project? SCORE
2.5
1.5
1.5
1.5
1.5
1.5
……….
COUPLES
……….
5
1i
ii fw
Manual Validation
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Possible Configurations
f1
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Possible Configurations
f1 +f2+ f3
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Possible Configurations
f1 +f2+ f4
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Possible Configurations
f5
(Baseline)
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Apache
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
12 14 16 18 20 22
Pre
cisi
on
Number of newcomer‐mentor pairs
PostgreSQL
f1 f1 +f2+ f3 f1 +f2+ f4 f5 (Baseline)
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18 19 20 21 22 23 24
Pre
cisi
on
Number of newcomer‐mentor pairs
Apache
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
12 14 16 18 20 22
Pre
cisi
on
Number of newcomer‐mentor pairs
PostgreSQL
f1 f1 +f2+ f3 f1 +f2+ f4 f5 (Baseline)
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
23 25 27 29 31 33 35 37 39 41
Pre
cisi
on
Number of newcomer‐mentor pairs
FreeBSD
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100%
24 26 28 30 32 34 36 38 40 42 44 46 48
Pre
cisi
on
Number of newcomer‐mentor pairs
Python
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100%
30 32 34 36 38 40 42
Pre
cisi
on
Number of newcomer‐mentor pairs
Samba
RQ1: How can we identify mentors from the past history of a software project?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
23 25 27 29 31 33 35 37 39 41
Pre
cisi
on
Number of newcomer‐mentor pairs
FreeBSD
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100%
24 26 28 30 32 34 36 38 40 42 44 46 48
Pre
cisi
on
Number of newcomer‐mentor pairs
Python
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100%
30 32 34 36 38 40 42
Pre
cisi
on
Number of newcomer‐mentor pairs
Samba
USEFUL FACTORS FOR MENTORS IDENTIFICATION
0.5*f1 + 0.25*f2 + 0.25*f4
0.5*f1 + 0.25*f2 + 0.25*f3
f1
RQ2: To what extent would it be possible to recommend mentors to newcomers joining a software project?
RQ2: To what extent would it be possible to recommend mentors to newcomers joining a software project?
RQ2: To what extent would it be possible to recommend mentors to newcomers joining a software project?
YODA make it is possible possible to recommend
Mentors
Why don’t just using Top Committers?
Why don’t just using Top Committers?
Why don’t just using Top Committers?
Not all Committers Are Good Mentors
Questions Asked: - Done/received mentoring
- Perceived importance of mentoring
- What makes a good Mentor
Surveying Projects Developers
Sent to 114 Subjects…
FreeBSD
Postgre- SQL
Python
Apache
Samba .....
37
..... 37
..... 15
..... 23
..... 23
Obtained Answare…
FreeBSD
Postgre- SQL
Python
Apache
Samba
-
92%
58%
8%
42%
0% 20% 40% 60% 80% 100%
Did mentoring?
Had a mentor?
YES NO
Done/received mentoring?
92%
58%
8%
42%
0% 20% 40% 60% 80% 100%
Did mentoring?
Had a mentor?
YES NO
Done/received mentoring?
Yes, I received Mentoring. My mentor was…
Yes, I did mentoring…
>
18%
36%
45%
0%
0%
33%
56%
11%
0%
0%
0% 20% 40% 60%
Very important
Important
Neutral
Not important
Useless at all
Effect of mentor Effect on newcomer
Perceived importance of mentoring
18%
36%
45%
0%
0%
33%
56%
11%
0%
0%
0% 20% 40% 60%
Very important
Important
Neutral
Not important
Useless at all
Effect of mentor Effect on newcomer
Perceived importance of mentoring
18%
36%
45%
0%
0%
33%
56%
11%
0%
0%
0% 20% 40% 60%
Very important
Important
Neutral
Not important
Useless at all
Effect of mentor Effect on newcomer
Perceived importance of mentoring
18%
36%
45%
0%
0%
33%
56%
11%
0%
0%
0% 20% 40% 60%
Very important
Important
Neutral
Not important
Useless at all
Effect of mentor Effect on newcomer
Perceived importance of mentoring
Is very important that mentor share knowledge with a mentee…
19%
42%
38%
0%
0% 10% 20% 30% 40% 50%
Experience
Communication skills
Project knowledge
Others
What makes a good Mentor
19%
42%
38%
0%
0% 10% 20% 30% 40% 50%
Experience
Communication skills
Project knowledge
Others
What makes a good Mentor
19%
42%
38%
0%
0% 10% 20% 30% 40% 50%
Experience
Communication skills
Project knowledge
Others
What makes a good Mentor
My first Mentor had a very strong and technical background
Conclusion
Conclusion
Conclusion
Conclusion
Conclusion
Future Work