Post on 11-May-2015
transcript
Supporting multimodal media recommendationand annotation using social network analysis
Adam Rae
a.rae@open.ac.uk
Supervisors Stefan Rüger, Suzanne Little, Roelof van ZwolDepartment The Knowledge Media InstituteStatus Full TimeProbation Viva AfterStarting Date October 2007
Research Hypothesis
By analysing and extracting information from the social graphs de-scribed by both explicit and implicit user interactions, like thosefound in online media sharing systems like Flickr1, it is possibleto augment existing non-social aware recommender systems andthereby significantly improve their performance.
Large scale web based systems for sharing media continue to tackle the problemof helping their users find what they are looking for in a timely manner. To dothis, lots of good quality metadata is required to sift through the data collectionto pick out exactly those documents that match the information need of theuser. In the case of finding images from the online photo sharing website Flickr,this could be from over 4 billion examples. How can we help both the systemand the user in enriching the metadata of the media within the collection inorder to improve the experience for the user and to reduce the burden on theunderlying data handling system? Can modelling users, by themselves andwithin the context of the wider online community help? Can this modeling beused to improve recommender systems that improve the experience and reducecognitive burden on users?
Existing approaches tend to treat multimedia in the same way they havedealt with text documents in the past, specifically by treating the textual meta-data associated with an image as a text document, but this ignores the inherentlydifferent nature of the data the system is handling. Images are visual data, andwhile they can be described well by textual metadata, they cannot be describedcompletely by it. Also, the user cannot be ignored in the retrieval process, andlearning more about a user provides information to the system to tailor results totheir specific requirements. Users interact online, and these interactions form a
1http://www.flickr.com/
2010 CRC PhD Student Conference
Page 91 of 125
new type of data that has yet to be fully explored nor exploited when modellingusers.
The work presented here combines the mining of social graphs that occur inFlickr with visual content and metadata analysis to provide better personalisedphoto recommender mechanisms and the following experiment and its analysisare a major component in my overall thesis.
Interaction ScenarioIn order to address this research question, multiple experiments have been car-ried out, one of which I present here:
Envisage an incoming stream of photos made available to a user. Insystems of a scale similar to Flickr, this could be thousands of im-ages per second. Can a system that uses cues from the social, visualand semantic aspects of these images perform better than one thatuses the more traditional approach of using only semantic informa-tion, according to specifically defined objective metrics? How doesperformance vary between users?
An experiment was carried out that mines data from the social communities inFlickr, from the visual content of images and from the text based metadata anduses a machines learning mechanism to merge these signals together to form aclassifier that, given a candidate image and prospective viewing user, decideswhether the user would label that image as a ‘Favourite’2 - see Figure 1.
Related Work
The significant influence that our peers can have on our behaviour online hasbeen studied by researchers such as Lerman and Jones[3], and the particularinteraction that occurs between users and visual media in particular in thework of Nov et al.[4]and Kern et al[2]. Their insights into the importance ofunderstanding more about a user in order to best fulfil their information needsupports the hypothesis that this kind of information can be usefully exploitedto improve systems that try to match that need to a data set supported by socialinteraction. Here I extend their ideas by incorporating this valuable social datainto a complementary multimodal framework that takes advantage of multipletypes of data.
The use of social interaction features in the work of Sigurbjörnsson and vanZwol[7] and Garg and Weber[1] inspired my more comprehensive feature setand its analysis. Their findings that aggregating data generated from onlinecommunities is valuable when suggesting tags is important and I believe alsotransfers to recommendation in general as well as to the specific task of recom-mending images. In fact, I demonstrated this in previous work on social mediatag suggestion[6].
I use some of the human perception based visual features outlined in thework of San Pedro and Siersdorfer[5], as these have been shown to work wellin similar experimental scenarios and cover a range of visual classes. I extendthem further with a selection of other high performing visual features.
2A binary label Flickr users can use to annotate an image they like.
2010 CRC PhD Student Conference
Page 92 of 125
User BMember of urban animals group
User AHas tagged beaches before
Incoming stream of previously unseen candidate images
Potential Favourite Imagesfor User A
Potential Favourite Imagesfor User B
TextualUser information User information
Social
Trained Classifier
Feature Extraction
Visual
Figure 1: Diagram of the image classification system used with Flickr data.
Experimental Work
400 users of varying levels of social activity were selected from Flickr and their‘Favourite’ labelled images collected. This resulted in a collection of hundredsof thousands of images. To train my classifier, these images were treated aspositive examples of relevant images. I generated a variety of negative examplesets to reflect realistic system scenarios. For all photo examples we extractedvisual and semantic features, and social features that described the user, theowner of the photo, any connection between them as well as other behaviourmetrics. We then tested our classifier using previously unseen examples andmeasured the performance of the system with a particular emphasis on theinformation retrieval metric of precision at 5 and 10 to reflect our envisaged usecase scenario.
Results
An extract of the results from the experiment are shown in Table 1. They canbe summarised thus:
• It is possible to achieve high levels of precision in selecting our positiveexamples, especially by using social features. This performance increaseis statistically significantly higher than the baseline Textual run. Thesesocial signals evidently play a significant rôle when a user labels an imagea ‘Favourite’ and can be usefully exploited to help them.
• The value of individual types of features is complex, but complementary.The combined systems tend to perform better than the individual ones.
• It is far easier to classifier photos that are not ‘Favourites’ than those thatare, as shown by the high negative values. This can be used to narrowdown the search space for relevant images by removing those that areobviously not going to interest the user, thus reduing load on both theuser and the system.
2010 CRC PhD Student Conference
Page 93 of 125
System Accuracy + Prec. + Rec. - Prec. - Rec.Textual 0.87 0.48 0.18 0.88 0.97Visual 0.88 1.00 0.09 0.88 1.00Social 0.92 0.80 0.56 0.94 0.98
Textual+Visual 0.88 0.62 0.27 0.90 0.97Textual+Social 0.92 0.77 0.62 0.94 0.97Visual+Social 0.93 0.89 0.56 0.94 0.99
Text+Vis.+Soc. 0.93 0.84 0.62 0.94 0.98
Table 1: Accuracy, precison and recall for various combinations of features usingthe experiments most realistic scenario data set. Photos labelled as ‘Favourites’are positive examples, and those that are not are negative examples. Highernumbers are better.
• As is typical in this style of information retrieval experiment, we can trade-off between precision and recall depending on our requirements. As we areinterested in high precision in this particular experiment, we see that thecombination of the Visual+Social and Text+Visual+Social runsgive good precision without sacrificing too much recall.
References
[1] Nikhil Garg and Ingmar Weber. Personalized, interactive tag recommenda-tion for flickr. In Proceedings of the 2008 ACM Conference on Recommender
Systems, pages 67–74, Lausanne, Switzerland, October 2008. ACM.
[2] R. Kern, M. Granitzer, and V. Pammer. Extending folksonomies for imagetagging. In Workshop on Image Analysis for Multimedia Interactive Services,
2008, pages 126–129, May 2008.
[3] Kristina Lerman and Laurie Jones. Social browsing on flickr. In Proceedings
of ICWSM, December 2007.
[4] Oded Nov, Mor Naaman, and Chen Ye. What drives content tagging: thecase of photos on flickr. In Proceeding of the twenty-sixth annual SIGCHI
conference on Human factors in computing systems, pages 1097–1100, Flo-rence, Italy, 2008. ACM.
[5] Jose San Pedro and Stefan Siersdorfer. Ranking and classifying attractive-ness of photos in folksonomies. In WWW, Madrid, Spain, April 2009.
[6] Adam Rae, Roelof van Zwol, and Börkur Sigurbjörnsson. Improving tagrecommendation using social networks. In 9th International conference on
Adaptivity, Personalization and Fusion of Heterogeneous Information, April2010.
[7] Roelof van Zwol. Flickr: Who is looking? In IEEE/WIC/ACM Inter-
national Conference on Web Intelligence, pages 184–190, Washington, DC,USA, 2007. IEEE Computer Society.
2010 CRC PhD Student Conference
Page 94 of 125
The effect of Feedback on the Motivation of Software
Engineers
Rien Sach
r.j.sach@open.ac.uk
Supervisors Helen Sharp
Marian Petre
Department/Institute Computing
Status Fulltime
Probation viva After
Starting date October 2009
Motivation is reported as having an effect on crucial aspects of software engineering
such as productivity (Procaccino and Verner 2005), software quality (Boehm 1981),
and a project’s overall success (Frangos 1997). Feedback is a key factor in the most
commonly used theory in reports published on the motivation of software engineers
(Hall et al. 2009), and it is important that we gain a greater understanding of the effect
it has on the motivation of software engineers.
My research is grounded in the question “What are the effects of feedback on the
motivation of software engineers?”, and focuses on feedback conveyed in human
interactions. I believe that before I can focus my question further I will need to begin
some preliminary work to identify how feedback occurs, what types of feedback
occur, and the possible impact of this feedback.
Motivation can be understood in different ways. For example, as a manager you might
consider motivation as something you must maintain in your employees to ensure
they complete work for you as quickly as possible. As an employee you might
consider motivation as the drive that keeps you focused on a task, or it might simply
be what pushes you to get up in the morning and go to work.
Herzberg (1987) describes motivation as “a function of growth from getting intrinsic
rewards out of interesting and challenging work”. That’s quite a nice definition; and
according to Herzberg motivation is intrinsic to one’s self. Ryan and Deci (2000)
describe intrinsic motivation as “the doing of activity for its inherent satisfaction
rather than for some separable consequence” (Page 60).
Herzberg (1987) defines extrinsic factors as movement and distinguishes it from
motivation, stating that “Movement is a function of fear of punishment or failure to
get extrinsic rewards”. Ryan and Deci (2000) state that “Extrinsic motivation is a
construct that pertains whenever an activity is done in order to attain some separable
outcome”.
There are 8 core motivational theories (Hall et al. 2009) and some of the theories
focus on motivation as a “a sequence or process of related activities” (Hall et al. 2009)
called process theories, while others focus on motivation “at a single point in time”
(Couger and Zawacki 1980) called content theories.
2010 CRC PhD Student Conference
Page 95 of 125
As reported in a systematic literature review conducted by Beecham et al (2007), and
their published review of the use of theory inside this review in 2009 (Hall et al 2009),
the three most popular theories used in studies of motivation in Software Engineering
were Hackman and Oldman’s Job Characteristics Theory (68%), Herzberg’s
Motivational Hygiene Theory (41%), and Maslow’s Theory of Needs (21%)1.
Hackman and Oldman’s Job Characteristics Theory focuses on the physical job, and
suggests five characteristics (skill variety, task identity, task significance, autonomy,
and feedback) that lead to three psychological states which in turn lead to higher
internal motivation and higher quality work. Herzberg’s Hygiene Theory suggests that
the only true motivation is intrinsic motivation, and this leads to job satisfaction,
where extrinsic factors are only useful in avoiding job dissatisfaction.
One of the five key job characteristics in Hackman and Oldman’s theory is feedback.
Feedback is not explicitly mentioned in Herzberg’s Motivational Hygiene Theory, but
he notes that it is a part of job enrichment, which he states is “key to designing work
that motivates employees” (Herzberg 1987). However this is a managerial view point.
Software Engineers are considered to be current practitioners working on active
software projects within the industry. This includes programmers, analysts, testers,
and designers who actively work and produce software for real projects in the real
world.
From a management perspective, gaining a greater understanding of what motives
employees could prove invaluable in increasing productivity and software quality, and
from an individual perspective the prospect of being given feedback that motivates
you and makes your job more enjoyable and improves the quality of your work
experience could lead to a more successful and enjoyable work life.
My proposed research is divided into stages. In the first stage I plan to conduct
interviews and diary studies to identify the types of feedback in software engineering
and how feedback is experienced by software engineers. I then plan to conduct
additional studies to identify what impact this feedback has on software engineers and
how that impact is evident. Finally, I plan to observe software engineers at work to
see feedback in context, and to compare those observations to the information
gathered during the first two stages.
At the end of my PhD I hope to accomplish research that leads to a greater
understanding of what feedback is inside software engineering and how it is given or
received. Subsequently I wish to gain an understanding of how this feedback alters the
motivation of software engineers and how this manifests as something such as
behaviour, productivity or attitude.
1 The percentages are a representative of how many of 92 papers the theories were found to be
explicitly used in. There can be multiple theories used in any one paper, and the 92 papers were part of
a systematic literature review conducted by Hall et al (2007) sampling over 500 players.
2010 CRC PhD Student Conference
Page 96 of 125
References
B.W. Boehm, Software Engineering Economics, Prentice-Hall, 1981.
COUGER, J. D. AND ZAWACKI, R. A. 1980. Motivating and Managing Computer Personnel.
John Wiley & Sons.
S.A. Frangos, “Motivated Humans for Reliable Software Products,” Microprocessors and
Microsystems, vol. 21, no. 10, 1997, pp. 605–610.
Frederick Herzberg, One More Time: How Do You Motivate Employees? (Harvard Business
School Press, 1987).
J. Procaccino and J.M. Verner, “What Do Software Practitioners Really Think about Project
Success: An Exploratory Study,” J. Systems and Software, vol. 78, no. 2, 2005, pp. 194–203.
Richard M. Ryan and Edward L. Deci, “Intrinsic and Extrinsic Motivations: Classic Definitions
and New Directions,” Contemporary Educational Psychology 25, no. 1 (January 2000): 54-67.
Tracy Hall et al., “A systematic review of theory use in studies investigating the motivations of
software engineers,” ACM Trans. Softw. Eng. Methodol. 18, no. 3 (2009): 1-29.
Sarah Beecham et al., “Motivation in Software Engineering: A systematic literature review,”
Information and Software Technology 50, no. 9-10 (August 2008): 860-878.
2010 CRC PhD Student Conference
Page 97 of 125