Recommender Systems
Cristina [email protected]
Department of Digital EconomySurrey Business School, UK
Jannis [email protected]
Department of ManagementLondon School of Economics, UK
Personalization increasingly mediates the experience of users on the Web. Online platforms
and organizations use personalization services to retain users, achieve longer user or
customer engagement and, ultimately, higher profits. Cast in this light, personalization is a
ubiquitous modality by means of which organizations seek to structure interaction with their
users. Amazon, for instance, mediates the buying experience of its customers through
computational systems that advance recommendations concerning relevant products to buy
upon nearly every transaction. Similarly, Spotify uses the listening habits of its users to
recommend tunes which they may find relevant to listen. In a rather different context,
Facebook modulates its news feed to the interests of individual users by mapping each user’s
ongoing interaction with his/her network of other users, and Google famously personalizes its
search engine results, gathering, in turn, relevant information on the search habits of users.
Collaborative filtering recommender systems are one amongst a complex and differentiated
landscape of technologies of personalization. Such systems are used alone or, increasingly, in
combination with other systems to develop and implement so-called hybrid recommenders
(for a useful taxonomy see Burke and Ramezani 2011). The form of user mediation the
various recommender systems offer may slightly vary in procedures and types of data
required but their logic and operations remain largely similar (Adomavicious and Tuzhilin
2005). All these systems work with a steady collection of carefully structured data produced
by myriads of individual transactions or interactions and the continuous feedback from users.
This engineering of experience through which individual actions are first produced, then
tracked, inferred, and transferred over to users presupposes a set of standardized and
automated procedures which —paradoxical as it may sound— become the backbone of
personalization. User interaction has to be shaped along actions standardized enough to leave
a computable data footprint underlain by carefully crafted user models that make individual
users comparable or better commensurable with one another.
Famously conveyed by Amazon’s: “Customers who bought this item also bought (…)”
collaborative filtering was first implemented in 1993 as a Usenet news recommender system
called GroupLens. The system recorded user ratings of news articles, stored them into a user
profile database and once it had enough data was able to relate ratings to recommend other
articles that users might find relevant (Konstan et al. 1997). A second experiment was done
with the site MovieLens. There, users were asked to rate movies on a scale of 5-stars and
paired into groups of similar users, that is, users who had rated movies similarly. Soon after
that, MIT built Ringo, a music recommender system (Shardanand and Maes 1995).
Riedl and Konstan are credited to have built one of the first collaborative filtering
recommender systems. Collaborative filtering, as the name suggests, uses information from
groups (hence, the collaborative dimension) to filter relevant items to individuals. Quite aptly
described as “any mechanism whereby members of a community collaborate to identify what
is good and what is bad” (Riedl and Konstan 1999: 330-331), collaborative filtering embeds
also elements of information retrieval and filtering, and a large list of automated and often
unsupervised computational operations. The “collaborative” element resonates with the so
called “wisdom of the crowd”, the expectation that information produced by masses of people
is somehow more precise or relevant than information produced by experts (Surowiecki
2005). Recommender systems take these insights along a distinctive route that reflects partly
the quest for personalization and partly the technological rendition of online experience we
outlined above.
Recommender systems are largely automated systems that derive from the application of AI
methods to information filtering and techniques of data representation and inference that have
their roots in the expert systems of the 1980s (Jannach 2010). Collaborative filtering gathers
information on user interactions, although users seldom interact with each other in these
settings, they only interact with the system to provide relevant information in the form of
buying transactions and ratings. Users, therefore, do not collaborate as the name collaborative
filtering might suggest. Nowhere is a community or group to be found. Those systems
compute affinities between users by gathering the independent preferences of atomized
individuals. Collaboration is euphemistically deployed to refer to these statistically-mediated
comparisons of user ratings. In fact, users are most of the times unaware of contributing to
the development and working of recommender systems and, when they happen to be, they are
seldom fully aware of the ways in which their contribution occurs. Advances in AI and
machine learning amplify such trends. As these systems grow in complexity and automation
the risk of overemphasizing implicit assumptions on the basis of which the system operates
also increases. Take the two-stage approach to personalization of the deep neural network
system for YouTube recommendation illustrated by Covington, Adams and Sargin (2016).
Here the system does not need to rely on user generated ranking anymore but has itself
learned how to rank. The system is made by two sub-systems: one for the generation of
videos to be recommended and another one for ranking those videos. The first network uses
collaborative filtering as it takes a user’s YouTube activity history and filters a smaller
sample of items to be recommended. The second system, called the ranking network,
automatically assigns a score to each video by gathering large volumes of data both on videos
and users (see fig 1). It is not by accident that YouTube has been at the centre of media
scrutiny recently as its modus operandi seeking prolonged user engagement led the system to
recommend mostly crude or offensive content1.
Figure 1: The deep neural network recommender system for YouTube. The figure illustrates the system architecture with its recommendation flow and relative data sources. From left to right: a. the video database b. the first sub-system of collaborative filtering c. the second sub-system of ranking and d. the recommendations to users. (Adapted from Covington, Adams and Sargin 2016).
The basic assumptions of collaborative filtering are that (i) the history of individual
preferences together with (ii) the history of the preferences of similar individuals are better
predictors than experts or the derivation of user preferences from market segments to which
1 See for instance: https://www.theguardian.com/technology/2018/feb/02/how-youtubes-algorithm-distorts-truth https://www.theguardian.com/technology/2018/feb/02/youtube-algorithm-election-clinton-trump-guillaume-chaslot
users are assumed to belong in standard marketing practices. In other words, the system
assumes that if users shared similar preferences in the past they will also share similar
preferences in the future. This broadly means that suggestions are tightly coupled with past
preferences and the preferences of similar others as mapped and recorded by the system.
As a general rule, in all recommender systems the kind of information selected and the ways
it is structured into databases and user profiles are directly connected with the core
technology of the system. In collaborative filtering, user profiles are modelled as a list
containing the history and quality of their ratings (Ricci et al. 2011); other systems may have
different user models and thus require different kind of information (e.g. data on items or a
mix between data on items and user behaviour). Information can be gathered by explicit
actions such as buying, rating, liking, watching, listening, etc. but they can also be collected
by using implicit behaviour such as browsing, searching, or time spent on webpages, etc. The
denominations of explicit and implicit refer to the indicator of consumer preferences which
are explicit when the indicators stem from actions that can be straightforwardly linked to
preferences; when the system does not have indicators of preferences, or data are sparse, it
infers them by interpreting as preferences any implicit actions. Other systems may use data
derived from items or from items and users and gather additional data from third-party
organizations. It is relevant to note that even when the system uses item data, it does not
simply record information but rather carefully designs specific data formats so as to fit the
core technology of the recommender system. A good example of the increased sophistication
in information design is given by the different ways music personalization techniques work2.
In collaborative filtering, once the system is up and running with a rating database and active
user profiles, it needs to cluster users or items into groups of similar users or items. To do so,
collaborative filtering recommender systems use two different approaches: user-based or
item-based algorithms. The first is the simplest and relies on the idea that given a rating
database the distance between users is determined by each individual user’s ratings of the
same item. The second instead computes the distance between items on the basis of how
closely users who have rated these items agree.
2 Pandora’s music genome project and The Echo Nest, the music intelligence platform empowering Spotify recommender engine, are two hybrid recommenders that approach the same problem (music personalization) in an entirely different way. See https://www.pandora.com/about/mgp and http://the.echonest.com/
Figure 2: Illustrative example of item-item personalization algorithm.
With a user-based algorithm, users who rate items similarly form neighbourhood of users.
With an item-based algorithm, items that are rated similarly form neighbourhood of items.
The grouping of users or items (called sometimes nearest neighbours or peer users) is how
the system establishes segments, that is, groups of similar users. Differently from traditional
marketing, in recommender systems this happens automatically and under computational
rules. There are no real similarities between user-user or item-item; users or items are
grouped together and deemed similar on the basis of the affinities emerging from rating
patterns provided by large amounts of data. The “segments” so created by the automated
recommender system become groups of predictors: for every item y that user x has not seen,
a prediction is computed on the basis of the rating of the item y made by nearest neighbours.
The majority of recommender systems use variants of a weighted, k-nearest-neighbour
prediction algorithm (roughly synthesized in “how much a target user u will like a target item
i by first selecting a neighbourhood of other users with tastes most similar to that of u”, (see
Konstan and Riedl 2012a; 2012b).
Figure 3: Illustrative example of the prediction based on nearest neighbours: “for every item y that user x has not seen, a prediction is computed on the basis of the rating of the item y made by nearest neighbours”.
To automatically select neighbours, however, the system needs first to compute a measure of
similarity between users or items. The computation of similarity is a fundamental problem for
the success of the prediction. That is, the measure by which two or more users or items are
deemed similar by the system and grouped together needs to be as accurate as possible. There
is no consensus yet to which of the different measures applied to the construction of the
similarity function works best (Alaimo 2013, 2014). A common approach remains the
Pearson’s correlation coefficient (another is the vector cosine similarity measure). “It
computes the missing rating of user u according to the average value of ratings made by its
neighbours weighted by each of their degree of similarity with the user u” (Ricci et al. 2011).
Figure 4: Pearson’s correlation coefficient. It is used to compute “the missing rating of user u according to the average value of ratings made by its neighbours weighted by each of their degree of similarity with the user u”.
The coefficient is successful in factoring ratings so as to make users commensurable. If a user
rates movies only with 4 and 5 stars and another user instead deploys the whole scale from 1
to 5, the ratings of the two users need to be adjusted in order to allow comparison. However,
the coefficient cannot solve sparsity problems, it cannot factor out the difference between the
ratings of two users if the former has rated 500 movies and latter only 5. Also, the coefficient
cannot make any difference between ratings that concern popular movies (most people like
Star Wars anyway) and ratings that concern less popular movies (which are arguably more
indicative of similarity of taste). Two other problems concern size and time. Size has always
been a problem for the successful computation of predictions. If the group of similar users or
items is too small, it becomes impossible to compute good predictions but if the size of the
group is too large this may be because the threshold of similarity is too low and predictions
will be inaccurate. Time instead refers to the complexity of re-computing the whole model
once a user adds a new rating. For this last reason, item-based collaborative filtering approach
is considered a better one. In this approach, the item-item algorithm calculates the distance
between items according to how much two users agree. It is important to bear in mind that
here similarity is computed between ratings, it is the pattern of user rating behaviour overtime
that is analyzed and computed to make two or more items nearest neighbours. Therefore, in
this case the distance between items is pre-computable as it is dependent on thousands of
available ratings thus remaining relatively stable overtime.
Customization and Personalization
The ideas presented above indicate that personalization and recommender systems are closely
associated with the online, data-intensive environments in which most organizations currently
operate. However, personalization is more than a technical response to the ubiquity of data
that characterize our age. It is above all an organizational practice that seeks to modulate a
space of interaction between organizations and users or customers in an economic, cultural
and social context that is increasingly marked by the fragmentation of consumer needs and
the individualization of consumption. Placing personalization within the larger historical
context of customization gives a better appreciation of the origin of this organizational
practice and the ways it has evolved with the use of digital technologies.
Customization has historically emerged as the organizational antidote to the typified
consumer experience characteristic of mass production and the long-driven standardization of
products or services which meticulous specialization and far-driven economies of scale have
brought about (Chandler 1977; Lampel and Mintzberg 1996). Despite significant variation
which the model of mass production has been subject to over the last few decades (Pine
1993), a great deal of products and services are still produced under conditions that recount
the exigencies of low unit cost, achieved through specialization, economies of scale and
standardization. Seen in this light, customization has been a response to this one-size-fits-all
consumer experience associated with standardized products and services. It is an
organizational practice that seeks to alleviate some of the negative implications of a long-
driven standardization and expand the possibilities of consumer choice.
In reality, customization operates by designing a space of interaction with consumers
whereby the latter are claimed to have ample freedom for exercising their choice. For
standard market segmentation techniques, individuals are just singular instances of wider
market segments associated with class, demographic, educational or income attributes. Under
these conditions an individual cannot but be one among a large group of similar others with
whom s/he shares a predictable set of needs. Customization assumes that such a rather
wholesale method for dissecting taste distribution among large populations is not any longer
well attuned to the faster production cycles of modern organizations nor to the context of
free-will individuals that seem to characterize hypermodern societies. As a response to these
changes, marketing segmentation techniques have been redefined to conceive individuals as
active consumers driven by desires mostly associated to lifestyle and other expressive
attributes of contemporary ways of living.
It is true that such a shift in marketing practices seldom moves far beyond the market
segmentation techniques with which standardized products and services have always been
closely associated. In fact, such consumer space of choice is more fictional than real. In a
great deal of cases, the exercise of personal choice assumed by customization is linked to
products and services that are mass-produced and then mass-customized. The adaptation to
individual customers is still mediated by segmentation techniques, certainly more finely
differentiated, that place individuals into smaller target groups (Zwick and Denegri Knott
2009; Zuboff and Maxmin 2003). Yet, targeting individual consumer desires instead of
product needs signalled a shift of capital importance for the development of the modern
individual consumer which we find closely relates to personalization. In this regard,
customization and personalization can be linked to a broader culture of individualism and
commodious consumerism characteristic of post- or late-modern societies which they
variously reinforce (Bauman 2000; Lipovetsky 2005). The current fragmentation of consumer
tastes is thus a broader societal phenomenon (Anderson 2006; Beck 1992) which has been
certainly escalated by the attempt of organizations to grapple with social changes (Zwick and
Cayla 2011). By redefining the space of consumption as a fictional space of limitless
consumer choice, organizations have reinforced the increasing fragmentation of taste
promoting a standardized notion of hyper-individuality.
From individuals to data
Personalization as an organizational practice and recommender systems as a primary
technology of personalization have transformed the ways data and data-based systems
mediate human experience of consumption and the space of consumer choice (Kallinikos
1992, 2007; Manovich 2001). This was noted as far back as 1999 when, in The New Yorker,
Malcom Gladwell (1999) wrote:
The really transformative potential of collaborative filtering, however, has to do with
the way taste products—books, plays, movies, and the rest—can be marketed.
Marketers now play an elaborate game of stereotyping. They create fixed sets of
groups—middle-class-suburban, young-urban-professional, inner-city-working-class,
rural-religious, and so on—and then find out enough about us to fit us into one of
those groups. The collaborative-filtering process, on the other hand, starts with who
we are, then derives our cultural ‘neighborhood’ from those facts. And these groups
aren’t permanent. They change as we change … (Italics added)
If we follow this account, personalization seems to go much further than customization.
Against the “lazy, prejudice philosophy” of demographic profiling, as Riedl and Konstan
(1999: 113) call traditional marketing segmentation, recommender systems claim to offer an
unbiased account of individual user needs and desires. Recommended systems aspire to
champion the resurgence of “who we are” out of the coarse taxonomies of traditional
marketing but also out of the late techniques of target groups and life styles characteristic of
customized marketing. Once again, personalization operates by re-engineering the space of
consumption. The space of limitless consumer choice is given an interesting tweak by being
transformed into a data field that is assigned the status of facts. Yet as we have seen,
recommender systems engineer the space of user interaction to fine-tune it to practices of data
gathering, user profiling and computation. Data are fashioned as facts by black-boxing the
numerous technological operations sustaining them which are far removed from user
interface and thus from user awareness. As distinct from past practices of customization,
interaction is now staged between an individual user and a digital system (not an organization
or other users) and shaped by iterative feedback loops between (i) programmed behaviour
and an initial data set, (ii) suggestions and (iii) user reactions and feedback by which the
system learns and readjusts its outputs.
Cast in the digital medium, the space of consumer choice articulated by the organizational
practice of personalization transforms traditional assumptions concerning individuals. On
most counts, personalization is the process of inference of what individuals and small taste
groups are or would like to be based on the clustering of data that are supposed to stand for
their actual preferences. There are few other cultural, social or personal references to which
the identity of individuals and groups is related. In the context of personalized
recommendations there are often no reference to data on actual buying behaviour but only
ratings and other clicking-related behaviours (Alaimo and Kallinikos 2017).
A critical look at these practices suggests that the practice of personalization is reductive in
ways that undermine any genuine concern for persons as unique cultural individuals.
Contrary to what Malcom Gladwell seems to suggest in the quote above, personalization -
mediated in recommender systems - is not concerned with persons but data. It compiles
profiles of individuals or taste groups out of digital marks (data) and what these last are
engineered to represent. Disturbing as it may seem, users online are seldom individuals in the
literal sense of the world. They are not real-world persons who exercise choice as the
outcome of their unique make-up of life experiences. They are rather data aggregates which
are put together on the basis of specific computational models (or profiles) of users imposed
by the core technology of the system (i.e. users are lists of such engineered operations such as
rating, listening, liking, etc.). This, in turn, conditions what kind of behaviour the system
needs to design so as to have the data required to perform adequately. As distinct from the
fictional lifestyles of late-consumerism and the rigid stereotypes of customization, individuals
online are just another digital item, constantly re-modulated, re-positioned and updated every
time a new click, like or rating is produced. Personalization systems help online consumers in
solving a problem that technology itself has caused. Overabundance of choices linked to
endless data-fields is a daunting scenario for users who therefore need to be aided by constant
personalized suggestions. In this respect personalization has many positive effects for
organizations as it is effectively correlated with higher user engagement and higher margins.
Personalization may also have some positive effects for users, as far as it helps users discover
new items, learn something new about themselves and overcome the anxiety of making
decisions (Anderson 2006). Yet, there is high price to pay for these functional gains. As
shown in this entry, whatever the implications of personalization, these need to be
appreciated within a larger time purview and the ways recommender systems encode and
engineer user experience.
Concluding remarks
In this entry we have reviewed the organizational practice of personalization by
deconstructing the ways collaborative filtering recommender systems work. Although
different recommender systems may be based on varying computational paradigms
(Adomavicius and Tuzhilin 2005; Burke and Ramezani 2011; Jannach, 2010; Ricci et al.,
2011) and may rely on several different data principles and algorithms, all of them work by
following a similar logic: (i) they need data on user and user behaviour (some gather data on
products as well); (ii) they need to construct and update a user model or profile whereby they
gather user preferences and past behaviour; (iii) they offer automated personalized
suggestions; (iv) they require continuous feedbacks from users to adapt and learn.
We have placed the emergence of personalization within the broader historical process of
customization and the quest of producing goods and services that are supposed to address the
distinctive needs of individuals and small groups. We have drawn attention to the mediating
properties of personalization systems and the current practice of data clustering and data-
based techniques that deeply impregnate personalization processes. Although the accuracy of
data-based techniques makes personalized services look as an empirically-grounded
mediation through which users can discover their own, allegedly true, needs and
predispositions, it is important to realize that there are no genuine individuals in these
systems, at least not in the sense we understand the term in real-life contexts (Alaimo and
Kallinikos 2016; 2017; Elmer 2004; Hildebrandt and Rouvroy 2011).
The datification of user experience which underlies personalization has a number of
consequences. Online involvement is heavily shaped by first translating individuals into user
profiles or computable models that render them steadily knowable entities. Any recommender
system actively fashions what an individual user or consumer is by creating models of users
which are highly dependent on the core technology-in-use. Such models constitute the
backbone for designing a set of clearly defined online interactions that enable the
computability of user preferences. In this respect, the individuality that personalization
constructs is no more than a changeable data profile. User models or profiles are assembled
out of strategies of attributing preferences to individuals through a complex journey of
technologizing experience whereby standardized expressions of individual behaviour —
clicking, liking or rating— are interpreted as expressions of taste and assessed by comparison
to a network of standardized behavioural expressions of others. Online individual consumers
are automatically fashioned by computational models as digital objects that are always
updatable and constantly in the making.
References
Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A
survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and
data engineering, 17(6), 734-749.
Alaimo, C. (2013). Technology of consumption on social shopping platforms: deconstructing
Similarity. EGOS: Montreal, Canada.
Alaimo, C. (2014). Computational consumption: Social media and the construction of digital
consumers. Doctoral dissertation, London School of Economics, London, UK.
http://etheses.lse.ac.uk/975/
Alaimo, C., & Kallinikos, J. (2017). Computing the everyday: Social media as data platforms. The
Information Society, 33(4), 175-191.
Alaimo, C., & Kallinikos, J. (2016). Encoding the everyday: The infrastructural apparatus of social
data. In Big data is not a monolith: Policies, practices, and problems, ed. C. Sugimoto, H.
Ekbia, and M. Mattioli, 77–90. Cambridge MA: MIT Press.
Anderson, C. (2006). The long tail: Why the future of business is selling less of more. Hachette Books.
Beck, U. (1992). Risk society: Towards a new modernity. London: Sage
Bauman, Z. (2000). Liquid modernity. New York: Wiley.
Burke, R. (2002). Hybrid recommender systems: Survey and experiments. User Modeling and User-
Adapted Interaction, 12(4), 331-370.
Burke, R., & Ramezani, M. (2011). Matching recommendation technologies and domains.
In Recommender Systems Handbook (pp. 367-386). Springer US.
Chandler Jr, A. D. (1993). The visible hand: The managerial revolution in American business.
Cambridge, MA: Harvard University Press.
Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for YouTube recommendations.
In Proceedings of the 10th ACM Conference on Recommender Systems (pp. 191-198). ACM.
Elmer, G. (2004). Profiling machines. Mapping the personal information economy. Cambridge, MA:
The MIT Press
Gladwell, M. (1999) The science of the sleeper, The New Yorker
https://www.newyorker.com/magazine/1999/10/04/the-science-of-the-sleeper
Hildebrandt, M., & Rouvroy, A. (Eds.). (2011). The philosophy of law meets the philosophy of
technology. London: Routledge.
Jannach, D. (2010). Recommender systems: An introduction. New York: Cambridge University Press
Kallinikos, J. (1992). The significations of machines, Scandinavian Journal of Management, 8/2: 113-
132.
Kallinikos, J. (2007). The consequences of information: Institutional implications of technological
change. Edward Elgar Publishing.
Konstan, J. A., Miller, B. N., Maltz, D., Herlocker, J. L., Gordon, L. R., & Riedl, J. (1997).
GroupLens: applying collaborative filtering to Usenet news. Communications of the
ACM, 40(3), 77-87.
Konstan, J. A., & Riedl, J. (2012a). Recommender systems: From algorithms to user experience. User
Modeling and User-Adapted Interaction, 22(1-2), 101-123
Konstan, J. A., & Riedl, J. (2012b). Recommended for you. IEEE Spectrum, 1-8
Lampel, J., & Mintzberg, H. (1996). Customizing customization. Sloan Management Review, 38(1),
21.
Lipovetsky, G. (2005). Hypermodern times. Cambridge: Polity.
Manovich, L. (2001). The language of new media. Cambridge, MA: The MIT Press
Mintzberg, H. (1979). The structuring of organizations. Englewood Cliffs: Prentice-Hall.
Pine, B. J. (1993). Mass customization: the new frontier in business competition. Cambridge, MA:
Harvard Business Press.
Ricci, F., Rokach, L., & Shapira, B. (2011). Introduction to recommender systems handbook.
In Recommender Systems Handbook (pp. 1-35). Springer US.
Riedl, J., & Konstan, J. (1999). Word of mouse: the marketing power of collaborative filtering.
Hachette UK.
Riedl, J., & Smyth, B. (2011). Introduction to special issue on recommender systems. ACM
Transactions on the Web, 5(1), 1-2
Shardanand, U., & Maes, P. (1995). Social information filtering: algorithms for automating “word of
mouth”. In Proceedings of the SIGCHI conference on Human factors in computing
systems (pp. 210-217). ACM Press/Addison-Wesley Publishing Co.
Surowiecki, J. (2005). The wisdom of crowds. New York: Anchor Books.
Wang, H., Wang, N., & Yeung, D. Y. (2015). Collaborative deep learning for recommender systems.
In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery
and Data Mining (pp. 1235-1244). ACM.
Zwick, D., & Cayla, J. (Eds.). (2011). Inside marketing: Practices, ideologies, devices. Oxford:
Oxford University Press.
Zwick, D., & Denegri Knott, J. (2009). Manufacturing customers: The database as new means of
production. Journal of Consumer Culture, 9(2), 221-247.
Zuboff, S., & Maxmin, J. (2003). The Support Economy: How corporations fail individuals and the
next episode of capitalism, London: Allen Lane.