Predicting programmers' personality via interaction ... · 80 such as MMPI-2 (Butcher, 1989), Big...

Predicting programmers' personality via interaction behaviourwith keyboard and mouseIftikhar A Khan, Osman Khalid, Waqas Jadoon, Rafi Us. Shan, Abdul Nasir

This exploratory research was conducted to study the relationship between Big Fivepersonality measurement scale and the interaction behaviour of the programmers withkeyboard and mouse to examine the possibility of creating a computer based objectivepersonality measurement scale. A field study with two analyses (N = 20), (N = 12) and alab study (N = 15) were conducted where N is the number of participants who participatedin the research. In the field study, interaction data were collected during normal PC useover several days. In the laboratory study, participants worked on a programming taskwhile their interaction with keyboard and mouse was being recorded in background. All ofthe participants rated their personality online and uploaded the data for analysis. Resultsfound inconsistent behaviour of all of the personality traits except ‘activity level’ in allstudies and thus suggested that a programmer’s ‘activity level’ can be predicted fromhis/her interaction behaviour with keyboard and mouse. This prediction will help indifferentiating good programmers from not so good programmers objectively.

PeerJ PrePrints | https://dx.doi.org/10.7287/peerj.preprints.1183v1 | CC-BY 4.0 Open Access | rec: 17 Jun 2015, publ: 17 Jun 2015

PrePrin

ts

2 Predicting Programmers' Personality via Interaction Behaviour with Keyboard & Mouse

3 Iftikhar Ahmed Khan, Osman Khalid, Waqas Jadoon, Rafi Us Shan, Abdul Nasir Khan

4 (iftikharahmed, osman, waqas_Jadoon, shan, anasir)@ciit.net.pk

5 COMSATS Institute of Information Technology, TOBE Camp Abbottabad, Pakistan

6 ABSTRACT

7 This exploratory research was conducted to study the relationship between Big Five personality

8 measurement scale and the interaction behaviour of the programmers with keyboard and mouse to

9 examine the possibility of creating a computer based objective personality measurement scale. A

10 field study with two analyses (N = 20), (N = 12) and a lab study (N = 15) were conducted where

11 N is the number of participants who participated in the research. In the field study, interaction data

12 were collected during normal PC use over several days. In the laboratory study, participants

13 worked on a programming task while their interaction with keyboard and mouse was being

14 recorded in background. All of the participants rated their personality online and uploaded the data

15 for analysis. Results found inconsistent behaviour of all of the personality traits except ‘activity

16 level’ in all studies and thus suggested that a programmer’s ‘activity level’ can be predicted from

17 his/her interaction behaviour with keyboard and mouse. This prediction might help in

18 differentiating good programmers from not so good programmers objectively.

19

20 Keywords: Programmers, Personality, Keyboard, Mouse, Interaction, Behaviour, Estimation,

21 Measurement

22 1. INTRODUCTION

23 The study of personality is one of the most active areas in the field of psychology (Burger, 2010).

24 One of the most agreed definition of personality is: “consistent behaviour patterns and

25 intrapersonal processes originating within the individual” (Burger, 2010). From definition it can

26 inferred that people have different personalities like some people make friends easily (extroverts)

27 others prefer to be alone (introverts). Similarly, some people are more business oriented while

28 others are service oriented. Therefore, studying personality is significant to identify the differences

29 among people behaviours. This implies that studying various human characteristics can help other


PrePrin

ts

30 humans to decide how to deal with a particular personality characters (Burger, 2010). Moreover,

31 these characteristics can help in marketing products (Nunes et al., 2012), finding effects of

32 personality on peoples’ own economy (Phau et al., 2009) and finding the relation between

33 personality and health (McCool, 2008) etc. In order to improve the human-human or human-

34 machine interaction as well as to enhance the productivity, the knowledge about various

35 personality features has gained much significance.

36 Various personality measurement instruments like (Butcher, 1989), (McCrae and John, 1992),

37 (Cattell, 1989) etc. can be used to collect knowledge about the personality. Personality is a

38 combination of traits and states that can be compared with already formulated personality

39 categories (Cohen et al., 2012). There are various techniques employed by researchers to measure

40 personality, such as MMPI-2 (Butcher, 1989), Big Five IPIP (McCrae and John, 1992), 16 PF

41 (Cattell, 1989), etc. Readers can find a detailed overview about personality measurement in (Cohen

42 et al., 2012) and (Boyle, 2008). All of these measuring instruments require self-reporting. There

43 are various versions of self-reporting scales like the one which require respondents to rate more

44 items and some which require to rate less items. The more items a scale contain, more reliable will

45 be the results. However, several studies showed that shorter versions were highly correlated with

46 full versions (Donnellan et al., 2006). Shorter versions tend to take less time as compared to the

47 versions which have more items. However, even shorter versions require allocation of time on part

48 of the participants. In addition, self-reports pose various validity and reliability problems (Stone

49 et al., 1999).

50 An alternative solution is automated measurement of personality that automatically measures a

51 user’s personality while doing his/her routine tasks. There is very little research on this approach.

52 Some examples include Heckmann who designed General User Model Ontology (GUMO) to store

53 users’ profile and to represent their psychological information through different kind of

54 technologies like semantic web and userML (User Model Mark-up Language) (Heckmann, 2005).

55 Personality recognizer from linguistic and conversational cues were also introduced (Mairesse et

56 al., 2007). There is also research available that adopted the personality measurement scale on the

57 computers (offline as well as online) but participants still needed to self-report these instruments.

58 An example of such adopted instrument is a website by Goldberg and Saicoer1.

1 http://ipip.ori.org/


PrePrin

ts

59 This research adopted the automated instrument to develop a behaviour based measurement of the

60 personality. This research supports the notion that personality can be expressed from behaviour.

61 For example, Brebner et al. (Brebner, 1983) showed that a change in visual stimulation caused a

62 higher level of motor activity in extroverts. Geen et al. (Geen, 1984) noted that extroverts prefer

63 more intense level of music as compared to introverts in a learning task. Extroverts interaction

64 time with the interfaces were quicker as compared to introverts (Saati et al., 2005). In this paper,

65 we explore the possibility that such interactive behaviours might also be visible through a keyboard

66 and mouse. As compared to other personality measurement techniques, a dispositional approach

67 is adopted to measure personality via usage of keyboard and mouse along with “Big Five” model

68 to describe personality characteristics.

69 Empirical studies and analysis performed on the gathered data indicated that it is possible to predict

70 some of participants’ personality traits from the interactive behaviour of computer programmers

71 with keyboard and mouse and thus forming the main scientific contribution of this paper. The rest

72 of the paper is organized as follows. In Section 2, we present the state of the art literature review.

73 A brief overview to Big Five Personality Measurement Model is provided in Section 3. Section 4

74 presents the field study we conducted in our work, whereas in Section 5, we present the conducted

75 laboratory study. Section 6 presents active windows analysis, and a discussion is provided about

76 programmer’s activity level as well as the proposed algorithm in Section 7. Finally, Section 8

77 presents the limitations of the proposed work along with conclusions and future work.

78 2. RELATED WORK

79 The task of personality measurement is complex. Various personality measurement scales exist,

80 such as MMPI-2 (Butcher, 1989), Big Five IPIP (McCrae and John, 1992), 16 PF (Cattell, 1989).

81 Such scales measure personality from a particular dimension. For example, in order to assess

82 biological-based traits, various electronic equipment like EEG or GSR (Heller, 1993) is used. For

83 subjective measurement of personality, subjective self-reporting scales like Big Five Factor

84 models (McCrae and John, 1992) are used. Alternatively, for objective measurement of

85 personality, objective scales like Objective Analytic Battery (OAB) (Cattell, 1955) is used.

86 Additionally, some scales, like multidimensional personality questionnaire is used to improve

87 multicultural effectiveness of the scales to measure personality (Van Der Zee and Van

88 Oudenhoven, 2000). All of these approaches are summarized under approaches like


PrePrin

ts

89 psychoanalytic approach, the Freudian approach, Non Freudian approach, the humanistic

90 approach, the cognitive approach as well as Behavioural/Social approach and have been discussed

91 in detail by Burger (Burger, 2010).

92 Personality depicts the habits of living beings specifically humans (Burger, 2010). This means that

93 people learn to behave and respond in a certain way and their responses for most of the times is

94 predictable. As discussed before, psychologists have developed various instruments to measure

95 these behaviours. These behaviours are then matched with a standard set of agreed upon

96 behaviours and are termed as personality (Cohen, 1988). However, significant time and effort is

97 required towards administering and managing these scales (Boyle, 2008). In addition, according

98 to Boyle (Boyle, 2008), there are very limited actual tests of personality and most of the scales

99 according to him are “self-report scales or reports of others’ rating scales”.

100 To overcome difficulties with self-reporting (as discussed in introduction) and to reduce

101 management and administrative issues, researchers devised various methods. In the present era of

102 computing, various researchers diverted their efforts towards adoption of personality measurement

103 scale for computers. Some studies, such as (Nunes et al., 2012), attempted to standardize the

104 personality information across applications. The aforementioned studies took human

105 psychological aspects into consideration while making decisions by the use of XML (Extensible

106 Mark-up Language). In addition Zhang et al. (Liang et al., 2012) presented a framework to

107 synthesize 2D cartoons and to depict their personality. People now are also able to take paper less

108 personality tests via website2. Such methods of evaluation might have reduced the managerial and

109 administrative issues, however, these tests are still based on self-reporting. Another issue is of data

110 privacy and handling as data access and control is not in the hand of every researcher and is mostly

111 restricted to the organization who own these scales. The researchers using such scale need to

112 convert textual results into their own format in order to do any analysis on the data. To overcome

113 these problems a more autonomous approach to measure personality is required.

114 Various automating activities related to personality with the help of computers exist. For example,

115 an interactive story telling system devised by Su et al. (Su et al., 2005) utilized hierarchical fuzzy

116 rule based system to facilitate the personality and emotion control of the dynamic story character.

2 http://ipip.ori.org/


PrePrin

ts

117 Similarly, Chittaro and Serra [23] presented a system for realistic behavioural programming of

118 virtual characters and based the characters’ programming on personality and probabilistic

119 automata (Chittaro and Serra, 2004). Various researchers such as [24], experimented on imparting

120 personality on virtual computer generated characters (Kshirsagar, 2002). One the same note,

121 Rousseau et al. [25] proposed a model to be used by intelligent automated actors. These actors in

122 turn were able to improvise their behaviour to interact with the users in a multimedia environment

123 (Rousseau, 1996). There is also research on personality and how it is related to online buying

124 behaviour (Bosnjak et al., 2007). For instance, the work presented in [26] imparts various

125 behaviours and personalities to computer characters by using different computational and

126 probabilistic models. However no research in authors’ knowledge addresses autonomous objective

127 prediction of personality of computer users in general and specifically programmers.

128 Programmers and computer users’ personality is being researched to improve their productivity

129 and to find right person for the right job. For example, Capretz (Capretz, 2003) reviewed empirical

130 studies on personality types of software engineers. Capretz concluded that software engineers are

131 significantly related to various personality types and are more likely to be sensing and thinkers

132 (STs), or Thinking and Judgmental (TJs), or Intuitive and thinkers (NTs) (Capretz, 2003). Software

133 professionals should be assigned roles from which it is evident that they have suitable personality

134 to carry out such tasks (Acuna et al., 2006), (Da Cunha and Greathead, 2007). Another example

135 of software professional personality should be considered while they work comes from (Salleh et

136 al., 2010) which found personality to be significantly related with pair programming students’

137 academic performance (Salleh et al., 2010).

138 In our proposed work, we analyse the behaviour of users on most commonly used devices, such

139 as keyboard and mouse. For computer users generally and for software programmers specifically

140 the keyboard and mouse are the most common devices for input and output. The keyboard and

141 mouse behaviour is already used by various researchers to predict moods of computer users. For

142 instance, in (Khan et al., 2013) and (Epp et al., 2011) the authors argued that it is possible to predict

143 some of the personality traits of programmers by recording their behaviour with keyboard and

144 mouse. In this paper, we utilized the Big Five Model of personality measurement. The next section

145 discussed in details the Big Five Model and the reasons why we selected this personality

146 measurement scale.


PrePrin

ts

147 3. BIG FIVE PERSONALITY MEASUREMENT MODEL

148 Although there is a difference of opinion among researchers in how to measure and what to

149 measure in a person’s personality, but still it is agreed that factor analytic properties of personality

150 are consistent (Burger, 2010). Various investigations on dynamic data like: (Digman, 1990),

151 (Goldberg, 1992) and (McCrae and Costa Jr, 1997) found that five basic dimensions of personality

152 are more or less consistent. These five dimensions of personality are so extensively used by

153 researchers that now it is being referred to as Big Five Model of personality measurement (Burger,

154 2010), (Boyle, 2008). The most commonly used names of the five dimensions are Neuroticism,

155 Extraversion, Openness, Agreeableness, and Conscientiousness. As the authors Boyle et al.

156 (Boyle, 2008) stated that in the construction of the Big Five scale, researches have not only utilized

157 the theory but also used empirical factor analytic inputs.

158 Such approach of defining traits is also the most common approach implemented by computing

159 scientists to measure personality via computers (Nunes et al., 2012). According to Nunes et al.,

160 trait approach highlights clear psychological differences among individuals and is easier to

161 categorize, organise, and to implement. Because of its wider and universal use, we considered Big

162 Five personality model in our paper for measuring personality of the participants.

163 Big Five personality measure contains five main traits and each main trait is further divided into

164 six sub-trait thus forming a total of 35 traits including the main trait. This specific 35 trait

165 personality questionnaire is called as NEO PI-R, a revised version by Costa and McCrae (Costa

166 and MacCrae, 1992). The original scale contains 300 items and take on average 30-40 minutes to

167 complete the test. The short version of scale contains 120 items and could take 10-20 minutes to

168 complete. The original scale is more reliable than the shorter one, though, shorter version also

169 meets professional standards of reliability (Costa and MacCrae, 1992). The test can be taken at the

170 following website3. IPIP Big Five scale main traits, sub-traits, and few of the characteristics are

171 shown in the Table 1 below.

172 Table 1: Big Five personality traits, sub-traits, and some of the related characteristics adopted

173 from Burger (Burger, 2010)

3 http://www.personal.psu.edu/faculty/j/5/j5j/IPIP/


PrePrin

ts

http://www.personal.psu.edu/faculty/j/5/j5j/IPIP/

Main Trait Sub-trait Some Characteristics

FriendlinessGregariousness

Assertiveness

Activity Level

Excitement Seeking

Extraversion

Cheerfulness

Sociable versus retiring, Fun-loving versus sober

Affectionate versus reserved, Energetic versus lazy,

Hurried versus slow, Aggressive versus ambitionless,

Active versus inactive

Trust

Morality

Altruism

Cooperation

Modesty

Agreeableness

Sympathy

Soft-hearted versus ruthless

Trusting versus suspicious

Helpful versus uncooperative

Self-Efficacy

Orderliness

Dutifulness

Achievement Striving

Self-Discipline

Conscientiousness

Cautiousness

Well organized versus disorganized

Careful versus careless

Self-disciplined versus weak willed

Anxiety

Anger

Depression

Self-Consciousness

Immoderation

Neuroticism

Vulnerability

Worried versus calm

Insecure versus secure

Self-pitying versus self-satisfied

Imagination

Artistic Interests

Emotionality

Adventurousness

Intellect

Openness to

Experience

Liberalism

Imaginative versus down-to-earth

Preference for variety versus preference for routine

Independent versus conforming

174

175 These studies involve collection of the participants’ personality as well as their interactive

176 behaviour with keyboard and mouse. The next sections will present two studies and three analyses

177 performed in this paper. The results are presented in the corresponding study sections.


PrePrin

ts

178 4. STUDY I – FIELD STUDY

179 4.1. Material

180 Most of the material collected and used in this study was also used in (Khan et al., 2013) to measure

181 mood of computer users from keyboard and mouse interaction. The basic idea of the experiment

182 was to record all keyboard and mouse related interaction of a user in log files. Therefore, we

183 developed a software application to record all the user interaction into log files. A mood rating

184 dialog was also designed to appear after every 20 minutes of logging to get participants’ mood

185 response. However, to reduce the concerns of participants over their privacy and security issues

186 various functionalities were developed in the application, such as:

187 Pause logging: Participants were able to pause application from logging for 5 and 10 minutes as

188 well as for custom specified period (in minutes) by clicking on the relevant menu. This option was

189 helpful for participants who wanted to do private tasks like bank transaction, etc.

190 Exit logging: Participants were also able to exit the application. Participants were instructed to

191 restart the application when they feel so. They were also told that application will restart itself

192 automatically on next computer boot.

193 Categorical Storage of keys: Some keys of the keyboard were divided into 4 categories namely:

194 ‘capital alphabets’, ‘lowercase alphabets’, ‘numeric keys’ and ‘special characters’. The keyboard

195 interaction were classified and recorded according to one of the categories instead of recording

196 original keys. In addition, mouse button events were recorded ‘Right Button’ or ‘Left Button’

197 depending on the right or left button click event. This mechanism further built the trust of

198 participants to continue using this application over several days.

199 4.2. Experimental setup

200 Departmental ethics committee namely CS-DARC (Computer Science Department Academic

201 Review Committee) reviewed and approved this experiment. A written consent for all studies was

202 obtained from the participants. The procedure of taking consent this way was approved by CS-

203 DARC. The developed application is run as a background application on the participants’

204 machines. The categorized keyboard interaction data described earlier were recorded under the

205 column ‘Category of the event’. In addition, the data for three more columns was recorded namely


PrePrin

ts

206 as: ‘Window name’, ‘Keyboard and Mouse Event’, and ‘Date/Time of the event’. Further

207 description of these three columns is as follows:

208 Window name: As modern day operating systems are designed to run different windows

209 (applications) at a time, the window name field was used to record active window. Names of the

210 document under work were not recorded. Only general software names under the use were

211 recorded like ‘Microsoft Power point’, ‘Mozilla Firefox’, etc. The data was recorded based on the

212 assumption that a person’s behaviour with the usage of keyboard and mouse will be different for

213 different applications. For example, frequent events of keyboard are expected while using an

214 editing application, chatting application or Integrated Development Environment (IDE) used by a

215 programmer during process of coding. Similarly, frequent mouse clicks are expected on a mouse

216 driven games like age of empires and rise of nations, etc.

217 Keyboard and Mouse event: When a key is pressed it initiates a ‘key down’ event. When released,

218 a ‘key up’ event is initiated. Similarly, ‘mouse click’ event is initiated on mouse button click. Such

219 data was used to record the three events: ‘Key Up’, ‘Key Down’ as well as mouse clicks (‘Left’ or

220 ‘Right’) button.

221 Date/Time of event: This column was used to record the date and exact time up to milliseconds

222 on the initiation of an event using the system date and time functionality.

223 A mood dialogue (Khan et al., 2013) was embedded in the log files as a marker to measure an

224 interaction. Application also provided the choice to participants to uninstall the application and zip

225 the log files, or to continue the experiment. Short form of Big Five IPIP-NEO4 (Goldberg, 1999)

226 instrument containing 120 items were used in this research. Participants were instructed to

227 complete the personality questionnaire online on IPIP website and email the results to the

228 researcher.

229 4.3 Participants

230 A total of 26 computer users participated in the mood study but only 20 consented to take

231 personality test as well. Therefore this study considered response of only 20 participants in total.

232 All of the participants were invited via personal contact or emails. The mean age of the

233 participants’ was 28.5 years and a standard deviation of 2.9 years. Participants’ computer


PrePrin

ts

234 experience mean was 5 years and standard deviation of 1.6 years. When asked from participants

235 how they rate themselves from the options as programmer, expert computer user, and a novice,

236 50% of them rated themselves as programmers. Expert computer users were 40% and 10% were

237 novices or beginners. There were only two females who participated in this study.

238 4.4. Results

239 4.4.1 Data preparation

240 Each participant continued logging for at least 5 days. Logging application was designed to create

241 a separate log file for each day with date and time stamp. All these file were merged together to

242 form a single file for a participant thus forming 20 different log files related to 20 participants. On

243 average, about a million lines of interaction of a participant with keyboard and mouse were

244 recorded. Therefore, a separate application was required to extract meaningful data from these

245 files. First step was to find a meaningful point around which data is analysed. This meaningful

246 point was identified as ‘mood rating’ embedded via mood rating dialog (Khan et al., 2013) in to

247 the log files. Average of behavioural data was calculated around every mood rating window4. An

248 interaction data of 3 minutes before mood rating and 3 minutes after mood rating were taken to

249 form a 6 minute window. The basic measures taken for each window were:

250 Self-reported mood: This was embedded in the log file. In the analysis part, this measure was

251 excluded because of no relevance to the study.

252 1. Total number of events (nEvents): Total events that took place within 6 minute window.

253 2. Average time between events (ATBE): Average of the time difference between occurrence of

254 two consecutive events

255 3. Standard deviation time between events (ATBE-STD): The standard deviation of ATBE

256 4. Total windows switched (WS): A switch by the participants from one window to another

257 window.

258 5. Number of backspace and delete key events (NBD): The events counter for only backspace

259 and delete keys. This is the counter of the possible errors committed by a user and the undo

260 action performed for those errors.

4 A window was formed around embedded mood rating in the log file and the interaction data before and after it


PrePrin

ts

261 6. Numeric and alphabet keys (NAK): Counter for numeric and alphabet keys

262 7. Mouse clicks (MC): Counter for total number of clicks (including right and left button clicks)

263 8. Other keys (OK): All other keys that include special keys, arrow keys, semi colons, colons etc.

264 Each participant’s average of the above measures and that of his/her rated personality was put into

265 an excel file for further refinement. There were N rows of entries for each participant’s interaction

266 data, where N is the number of mood rating dialogs answered by participants. There was only 1

267 row of personality traits. Before averaging a participant’s interaction data over N rows, all the

268 events where interaction was less than 10 events in a window were filtered out. The reason was

269 that the participants might not be actively using keyboard and mouse but instead reading from

270 some website or busy in some other tasks. All events where difference of time between two events

271 was less than 50 milliseconds were also eliminated. The reason being that an experienced typist

272 on average cannot type a meaningful character of a sequence in less than 60 milliseconds (Card et

273 al., 1983). Considering an experienced typist, we reduced filter to 50 milliseconds just in case that

274 someone among the participants could be an experienced typist. In addition, all the events with a

275 difference of time more than 20 seconds were also eliminated on the same grounds of inactivity.

276 After required filtration of each participant data, all the interactions rows were averaged to a single

277 row. In addition to means, standard deviation of each interaction variables was also calculated.

278 4.4.2. Analysis

279 First step toward analysis of this data was to carry out correlations analysis. As aim of this study

280 is to find out the relation between keyboard and mouse interaction behaviour and personality traits.

281 Therefore, a correlation analysis was conducted between interaction behaviour variables and

282 personality trait variables. The results revealed that various traits and sub-traits are significantly

283 correlated with interaction behaviour of participants with keyboard and mouse. The results are

284 presented from Table 2–Table 7 below.

285 Table 2: Correlations between Extroversion and its sub-traits with that of keyboard and mouse

286 behaviour

Interaction Variables Gregariousness Activity Level Excitement

Seeking

Cheerfulness

ATBE-Mean -0.44*


PrePrin

ts

ATBE-Standard Deviation -0.43* 0.44* -0.53*

NAK-Mean 0.40* 0.41*

NAK-Standard Deviation 0.41* 0.41*

OK–Mean 0.41*

OK- Standard Deviation 0.48*

287 Note: ** significant Correlation at 0.01 levels, * significant Correlation at 0.05 levels for N = 20.

288 Table 3: Correlations between Agreeableness and its sub-traits with that of keyboard and mouse

289 behaviour

Agreeableness Trust Morality Cooperation Sympathy

ATBE-Mean 0.41*

WS-Mean -0.56** -0.56** -0.52*

WS-Standard

Deviation-0.48*

NBD-Mean -0.51* -0.43*

NBD-Standard

Deviation-0.40* -0.40* -0.40*

NAK-Mean -0.45*

NAK-Standard

Deviation0.47*


291 Table 4: Correlations between Conscientiousness and its sub-traits with that of keyboard and

292 mouse behaviour

Conscientious

ness

Self-Efficacy Orderliness Dutifulne

ss

Cautiousn

ess

nEvents–Mean -.60** -.46*

WS-Mean -.42*

NAK-Mean .46*

MC–Mean .44*

MC–Standard Deviation .52*

OK-Mean -.40* -.43* -.55**


PrePrin

ts

OK-Standard Deviation -.56**


294 Table 5: Correlations between Neuroticism and its sub-traits with that of keyboard and mouse

295 behaviour

Neuroticism Anxiety Self-Consciousness Vulnerability

ATBE-Mean .43*

WS-Mean .50*

NAK-M .51*

MC-M -.40* -.53*

MC-Standard Deviation -.40* -.40* -.51* -.41*

296 Note: ** significant Correlation at 0.01 levels, * significant Correlation at 0.05 levels for N = 20.Table

297

298 Table: 6: Correlations between Openness to experience and its sub-traits with that of keyboard

299 and mouse behaviour

Openness to

experience

Artistic

Interests

Emotionality Intellect Liberalism

nEvents-Mean .43*

ATBE-Standard Deviation .41*

WS-Mean -.40* -.54* -.60**

WS-Standard Deviation -.48* -.62** -.58**


301 Correlations are considered of small, medium, and large size with r = 0.1, r = 0.3 to 0.49, and r =

302 0.5 or above, respectively (Cohen, 1992). According to the aforementioned criterion, all

303 correlations revealed in Tables 2–7 either have medium or large effects.

304 5. STUDY II – LAB STUDY

305 Various personality traits were found to be significantly correlated with the keyboard and mouse

306 interaction variables mentioned in section 4.4.1. While in study 1, the interaction of the participants

307 with keyboard and mouse were logged in normal working conditions, in the second study

308 programming tasks of about an hour duration were given to participants as task to minimize


PrePrin

ts

309 boredom and to reduce activity time. The idea was same to replicate the earlier findings with

310 different experimental setup and conditions. Research shows that personality of people mainly

311 remains constant over years and external conditions like time, environment etc. do not effect

312 personality of people (Shaffer, 2009). Therefore, it was hypothesized that interaction of

313 participants with keyboard may produce similar results as that of experiment 1. The material used,

314 preparation of the data, analysis and results are presented in the following sections

315 5.1. Material

316 The customized application used in study 1 with disabled mood dialogs were modified to log

317 keyboard and mouse interaction data. Original data was not recorded and instead categorical data

318 were recorded. Like previous study this study also measured participants’ personality by using

319 short form of IPIP-NEO Big Five personality instrument. In this study programming tasks

320 assigned to participants were to simulate a story (see section 5.2) in ALICE programming

321 language. ALICE is a free object oriented and simulation based language that is often utilized to

322 teach programming to children.

323 5.2. Experimental setup

324 Main differences from experiment 1 were in the setup of the experiment. A quiet room without an

325 external interference was reserved for the experiment. All of the participants were either

326 programmers or expert computer users. They were trained to program in ALICE. All participants

327 successfully completed four learning tutorials already present in ALICE documentation for

328 learning purposes. Two of the Aesop stories (‘the hare and the tortoise’ and ‘the cow and the frog’)

329 were assigned to participants to simulate. Music was played in background while participants were

330 performing tasks. The experiment was divided into two sessions with a gap of 10-20 minutes in

331 each session. The logging application was running in background to log keyboard and mouse

332 interaction during the entire duration of the experiment. Same variables as discussed in section 4.1

333 were also being logged in this study.

334 5.3. Participants

335 In this experiment, 15 participants were involved. All participants rated themselves as expert

336 computer users but with no experience of programming in ALICE. The participants mean age was


PrePrin

ts

337 21 years with standard deviation of 3.1. The mean experience of participants of computer usage

338 was 7.3 years and standard deviation of 2.77. There were no female participants in this study.

339 5.4. Results

340 5.4.1 Data preparation

341 Average of eight behavioural measures such as: (1) number of events, (2) average time between

342 events, (3) standard deviation of the time between events, (4) total windows switched, (5) number

343 of backspace and delete keys, (6) number of alphabetic and numeric keys, (7) number of mouse

344 clicks, and (8) number all other keys were calculated over the duration of the experiment. Before

345 calculating average of behavioural variables over the sessions, the data was filtered with filters

346 like: ‘time difference between the two events’ should be greater than 50 milliseconds and less than

347 20 seconds The reason of using these filters is already discussed in section 4.4.1. This data was

348 merged with personality data in an excel file forming a single row with first eight columns of

349 interaction data and last columns containing personality data.

350 5.4.2 Analysis II

351 Pearson correlation was conducted between the interaction data and personality data. Various traits

352 and sub-traits of personality were again found to be significantly correlated with that of keyboard

353 and mouse interaction behaviour. Only one result appears to be significantly correlated in both

354 studies – the standard deviation of average time between events and activity level with r=0.59 and

355 p<=0.05. The results from this study and the earlier one indicates that behavioural measures have

356 no relation with personality traits except activity level and standard deviation of average time

357 between events. Activity levels fall within the sub-traits of extraversion. It is related to the level

358 of excitement and energy within individuals (Whalen and Gates, 2007). In both studies activity

359 level was found to be positively correlated with that of standard deviation time between events.

360 To confirm this effect, another analysis was planned by utilizing the data from the experiment 1.

361 6. ACTIVE WINDOWS ANALYSIS-STUDY I

362 As the first experiment was field study with data collected from participants working at their

363 offices/homes etc. therefore participants interacted with various windows during the experiment.

364 One of the columns ‘Active windows’ (on which the participants were working on) was being


PrePrin

ts

365 recorded in the log files. Its purpose was to measure the keyboard and mouse interaction with

366 different applications. A skimming through these data revealed that participants interacted with

367 various different applications. There were various applications which do not need a lot of

368 interaction from participants like audio/video players, as well as Installation/Uninstallation

369 windows etc. The remaining applications were categorized as below:

370 1. Browsers

371 2. Word Editing

372 3. Messaging

373 4. Programming and Development

374 5. Games

375 Some of the windows/software were identifiable by their general name like Microsoft word and

376 Internet Explorer. However, some of the windows were not identifiable as they were tagged like

377 “unidentifiable window” or “False Window”. The reason was that these applications unlike various

378 other applications did not contain sub documents but one general name. For example a document

379 named ‘doc1’ will display a title like: ‘Microsoft Word – doc1’. Extracting the name of the

380 application without identifying the name of document becomes easy. However, some applications

381 like games were logged as ‘Unidentifiable Window’ to protect information related with

382 participants whereas applications without sub-documents with direct names were recorded as

383 ‘False Windows’

384 Keystroke analysis against ‘Unidentifiable windows’ and ‘false windows’ showed mouse clicks

385 and ‘other keys’ events dominated over all other events. As up, down, left, and right arrows were

386 included in ‘other keys’, therefore these windows were most likely to be games as usually games

387 use a lot of mouse clicks as well as direction keys. Similarly, use of alphabetic and numeric keys

388 was minimal compared to the aforementioned windows thus supporting the claim that the window

389 identifies some game being played. Only two of the entries after analysing the keys were not

390 clearly identifiable and therefore were left out of the analysis.

391 Different participants showed different frequency of use of the above mentioned five categories.

392 If total number of events by a participant for an application over five days were less than 1000

393 events, then this application was not included in the analysis. Browsers and Gaming applications


PrePrin

ts

394 were used by all the 20 participants included in this analysis. However, word editing applications

395 with events greater than 1000 were only used by 16 participants. Chatting/Messaging was used by

396 19 participants. Programming and development related applications were used by only 12

397 participants. The filters like: time difference of 50 milliseconds between two events and less than

398 20 seconds were also applied on this data.

399 6.1 Results

400 Correlation analysis was conducted between ‘standard deviation of average time of events’ on five

401 categories of applications separately with that of ‘activity level’. Only the one sub-trait of

402 extroversion ‘activity level’ was considered as this variable showed significant correlations in

403 earlier two studies. Results revealed that activity level was only significantly correlated with the

404 standard deviation of the average time between events that occurred on the programming

405 applications (r = 0.646, p = 0.023). The outcomes support the notion that generally the participants

406 who tend to be programmers have positive correlation with their activity level.

407 DISCUSSIONS-PROGRAMMERS’ ACTIVITY LEVEL

408 In this study, three types of computer users participated: (a) programmers, (b) experts, and (c)

409 novices. These ranking were participants’ own perceptions. On analysis of the participants’

410 keyboard and mouse interaction data from experiment 1, it appeared that all the users either were

411 programmers or were users who used computer frequently. All of the participants used word

412 browsers and game applications. Majority participants (a total of 19) used chatting/email

413 applications as well as 16 used word editing applications. Similarly, of all the participants were

414 experienced computer users’ with mean experience of 2 years or more. In the second study, all the

415 participants’ were trained of programming task and eventually carried out that programming task.

416 Therefore, it is safe to presume that finding of this study largely dealt with programmers. Results

417 from all of the analysis showed that it is possible to predict programmers’ activity level from their

418 interaction behaviour with keyboard and mouse.

419 Activity level is a sub-trait of extraversion. According to (Goldberg, 1999) a person with positive

420 activity level is one who can manage multi-tasking, target goals and achieve them without wasting

421 his time. These are all attributes that belong to good programmers. Researches like (Stolee et al.,

422 2011) and (Staurt, 2010) are some of the examples that discussed on the positive characteristics of


PrePrin

ts

423 the programmers. Therefore, the results of our research can provide a useful application to predict

424 good programmers vs. not so good programmers.

425 Programmers’ activity level might also be dependent on the extent of stress they face during

426 development. Nan and Harter et al. (Nan and Harter, 2009) demonstrated relationship between

427 schedule pressure and development activities in software development life cycle (SDLC). They

428 concluded in their paper that excessive pressure affect development performance negatively. An

429 indirect inference is that increased pressure may cause a programmer to rapidly press keyboard

430 keys, indicating high activity level on keyboard and mouse. Therefore, one of the findings of this

431 research is that the pressure to complete a task within a deadline may cause increase activity level

432 with the keyboard and mouse.

433 The variables having significant correlations were analysed to find what frequency of keyboard

434 interaction is related with which level of activity, where activity level is a participants’ personality

435 trait rating out of 100. Both analyses of experiment 1 revealed that in conditions where there is no

436 pressure on the programmers and when their activity level is equal to or above 60 then standard

437 deviation of difference between events time lies above 1600 milliseconds. In the pressure

438 development as in experiment 2, standard deviation time was reduced to 530 milliseconds for

439 programmers having activity level above 60. Considering these statistics, an algorithm to predict

440 programmers’ activity level is formulated as follows:

441 ALGORITHM

442 A general algorithm to measure a person’s activity level from his/her interaction behaviour with

443 keyboard and mouse is formulated. As evident from current research [Khan et. el, 2013] that this

444 algorithm will most likely to work on programming related application like I.D.Es’.

445 1: Begin Program

446 2: Let Ec a current keyboard or mouse interaction event

447 3: Let Ecp a previous keyboard and mouse interaction event. If this is the first event then Ecp = 0

448 4: Record programmers’ keyboard and mouse interaction over or at least for n minutes in a file

449 F where n >= 3

450 5: Set ‘previous event’ to null

451 6: For time t loop from t = 0.1 second to a finite time but greater than n minutes


PrePrin

ts

452 7: Scan ‘current event’ from file F

453 8: If (’ previous event’ is not null) then

454 9: Let TD a variables that will store time difference between two events

455 10: TD = absolute (Ecp - Ec)

456 11: Comment: The line below shows filter applied on the interaction data

457 12: If (TD > 50 ms) and (TD < 20s)

458 13: Add the difference into an array Ai

459 14: End If (TD > 50 ms) and (TD < 20s)

460 15: End If (there is previous event)

461 16: Put ‘current event’ in ‘previous event’

462 17: Scan file for next event and put it into ‘current event’

463 18: End for

464 19: Let SD = Calculate Standard deviation of all the elements in Ai

465 20: If (Development Environment is a Pressure Environment) and

466 21: If (SD > 530 milliseconds)

467 22: Predication “Programmer has high activity level and might be a good

468 programmer”

469 23: End if (SD > 530 milliseconds)

470 24: Else (Development Environment is not a Pressure Environment)

471 25: If (SD > 1600 milliseconds)

472 26: Predication “Programmer has low activity level and might not be a good

473 programmer”

474 27: End if (SD > 1600 milliseconds)

475 28: End if ((Development Environment == Pressure Environment) and

476 29: End Program

477

478 7. CONCLUSIONS, LIMITATIONS AND FUTURE RESEARCH

479 Based on the participants as well as results of study 1, it is evident that most of the personality

480 traits and sub-traits correlate with keyboard and mouse interaction behaviour. However, only

481 activity level was found to be positively correlated in both the experiments and three analyses.

482 Moreover, as these participants had more than two years of experience with computers as well as


PrePrin

ts

483 almost all participants showed significant usage of applications that need expertise, it can also be

484 safely concluded that these participants were either programmers or possessed programming skills.

485 The main conclusion of the research is that programmers’ personality specifically activity level

486 can be predicted from their interaction behaviour with keyboard and mouse events.

487 Like all empirical researches, this research also has some limitations. For example, sample

488 participants were mostly graduate and PhD students that limited the findings specific to only such

489 people. Another limitation was that these results are mostly representative of male population as

490 there were only two females in study 1 and no female in study 2. However, this limitation denotes

491 an overall trend of technology and engineering industry where female population is scarce. In

492 addition, 40% of the participants in both the studies were same. Therefore, there is a chance of

493 repetition in the observed correlations. However, as sampling was done in two different

494 environments, the aforementioned limitation might have negligible effect on overall results of this

495 research.

496 Various future possibilities exist from this research. This research laid a foundation for an

497 application to measure users’ personality (activity level) from their interaction behaviour with

498 keyboard and mouse. Further experiments to test each of five personality dimensions separately

499 with the keyboard and mouse interaction might enable researchers to detect personality with more

500 accuracy.

501 Acknowledgements

502 I am thankful to Dr. William-Paul Brinkman of Delft University of Technology, The Netherlands

503 under whose kind supervision, I have collected the data used in this paper.

504 8. REFERENCES

505 Acuna ST, Juristo N and Moreno AM. (2006) Emphasizing human capabilities in software development. 506 Software, IEEE 23: 94-101.507 Bosnjak M, Galesic M and Tuten T. (2007) Personality determinants of online shopping: Explaining online 508 purchase intentions using a hierarchical approach. Journal of Business Research 60: 597-605.509 Boyle GJM, G.; Saklofske, D. H.;. (2008) Personality measurement and testing: An Overview. In: Boyle 510 GJM, G.; Saklofske, D. H.; (ed) Personality and Social Sciences papers. Paper 294. London: SAGE, 511 1-29.512 Brebner J. (1983) A model of extraversion. Australian Journal of Psychology 35: 349-359.513 Burger JM. (2010) Personality: Wadsworth Cengage Learning.


PrePrin

ts

514 Butcher JND, W. G.; Graham, J. R.; Tellegen, A; Kaemmer, B;. (1989) The Minnesota Multiphasic Personality 515 Inventory-2 (MMPI-2): Manual for administration and scoring, Minneapolis: University of 516 Minnesota Press.517 Capretz LF. (2003) Personality types in software engineering. International Journal of Human-Computer 518 Studies 58: 207-214.519 Card KC, Moran TP and Newell A. (1983) The Psychology of Human-Computer Interaction, Hillsdale, New 520 Jersey: Lawrence Erlbaum.521 Cattell HB. (1989) The 16PF: Personality in depth: Institute for Personality and Ability Testing Illinois.522 Cattell RB. (1955) Handbook for the Objective-Analytic Personality Test Batteries:(including Adult and Child 523 OA Batteries): Institute for Personality and Ability Testing.524 Chittaro L and Serra M. (2004) Behavioral programming of autonomous characters based on probabilistic 525 automata and personality. Computer Animation and Virtual Worlds 15: 319-326.526 Cohen, Swerdlik ME and Sturman ED. (2012) Psychological Testing and Assessment: An Introduction to 527 Tests and Measurement: McGraw-Hill Education.528 Cohen RJM, P.; Nathanson, L. S.; Swerdlik, M. E.;. (1988) Personality Assessment: An Overview. 529 Psychological Testing: An Introduction to

530 Tests and Measurements. Mountain View, CA: Mayfield Publishing Co.531 Cohen; J. (1992) Statistical Power Analysis. Current Directions in Psychological Science 1: 98-101.532 Costa PT and MacCrae RR. (1992) Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor 533 Inventory (NEO FFI): Professional Manual: Psychological Assessment Resources.534 Da Cunha AD and Greathead D. (2007) Does personality matter?: an analysis of code-review ability. 535 Communications of the ACM 50: 109-112.536 Digman JM. (1990) Personality structure: Emergence of the five-factor model. Annual review of 537 psychology 41: 417-440.538 Donnellan MB, Oswald FL, Baird BM, et al. (2006) The mini-IPIP scales: tiny-yet-effective measures of the 539 Big Five factors of personality. Psychological assessment 18: 192.540 Epp C, Lippold M and Mandryk RL. (2011) Identifying emotional states using keystroke dynamics. 541 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 715-724.542 Geen RG. (1984) Preferred stimulation levels in introverts and extroverts: Effects on arousal and 543 performance. Journal of Personality and Social Psychology 46: 1303.544 Goldberg LR. (1992) The development of markers for the Big-Five factor structure. Psychological 545 assessment 4: 26.546 Goldberg LR. (1999) A broad-bandwidth, public domain, personality inventory measuring the lower-level 547 facets of several five-factor models. Personality psychology in Europe 7: 7-28.548 Heckmann D. (2005) Ubiquitous user modeling: IOS Press.549 Heller W. (1993) Neuropsychological mechanisms of individual differences in emotion, personality, and 550 arousal. Neuropsychology 7: 476.551 Khan IA, Brinkman W-P and Hierons R. (2013) Towards estimating computer users’ mood from interaction 552 behaviour with keyboard and mouse. Frontiers of Computer Science: 1-12.553 Kshirsagar S. (2002) A multilayer personality model. Proceedings of the 2nd international symposium on 554 Smart graphics. ACM, 107-115.555 Liang Z, Xiao J and Zhuang Y-t. (2012) Synthesizing style-preserving cartoons via non-negative style 556 factorization. Journal of Zhejiang University SCIENCE C 13: 196-207.557 Mairesse F, Walker MA, Mehl MR, et al. (2007) Using Linguistic Cues for the Automatic Recognition of 558 Personality in Conversation and Text. J. Artif. Intell. Res.(JAIR) 30: 457-500.559 McCool LC. (2008) The Health Behaviours of Exercise and Dietary Intake: Links with Personality and 560 Coping, in a student sample.


PrePrin

ts

561 McCrae RR and Costa Jr PT. (1997) Personality trait structure as a human universal. American psychologist 562 52: 509.563 McCrae RR and John OP. (1992) An introduction to the five-factor model and its applications. Journal of 564 personality 60: 175-215.565 Nan N and Harter DE. (2009) Impact of budget and schedule pressure on software development cycle time 566 and effort. Software Engineering, IEEE Transactions on 35: 624-637.567 Nunes MASN, Bezerra JS and de Oliveira AA. (2012) Personalityml: a markup language to standardize the 568 user personality in recommender systems. GEINTEC-Gestão, Inovação e Tecnologias 2: 255-273.569 Phau I, Sequeira M and Dix S. (2009) To buy or not to buy a.570 Rousseau D. (1996) Personality in computer characters. Proceedings of the 1996 AAAI Workshop on 571 Entertainment and AI/A-Life. 38-43.572 Saati B, Salem M and Brinkman W-P. (2005) Towards customized user interface skins: investigating user 573 personality and skin colour. Proceedings of HCI 2005 2: 89-93.574 Salleh N, Mendes E, Grundy J, et al. (2010) An empirical study of the effects of conscientiousness in pair 575 programming using the five-factor personality model. Proceedings of the 32nd ACM/IEEE 576 International Conference on Software Engineering-Volume 1. ACM, 577-586.577 Shaffer D. (2009) Social and personality development: CengageBrain. com.578 Staurt; A. (2010) 50 characteristics of a great software developer.579 Stolee KT, Elbaum S and Sarma A. (2011) End-user programmers and their communities: An artifact-based 580 analysis. Empirical Software Engineering and Measurement (ESEM), 2011 International 581 Symposium on. IEEE, 147-156.582 Stone AA, Bachrach CA, Jobe JB, et al. (1999) The science of self-report: Implications for research and 583 practice: Psychology Press.584 Su W-P, Pham B and Wardhani A. (2005) High-level control posture of story characters based on 585 personality and emotion. Proceedings of the second Australasian conference on Interactive 586 entertainment. Creativity & Cognition Studios Press, 179-186.587 Van Der Zee KI and Van Oudenhoven JP. (2000) The Multicultural Personality Questionnaire: A 588 multidimensional instrument of multicultural effectiveness. European journal of personality 14: 589 291-309.590 Whalen T and Gates C. (2007) A psychological profile of defender personality traits. Journal of Computers 591 2: 84-93.

592

593


PrePrin

ts

Date post:	15-Dec-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Predicting programmers' personality via interaction ... · 80 such as MMPI-2 (Butcher, 1989), Big...

Documents