Proceedings on Privacy Enhancing Technologies ; 2016 (4):237–254

Christoph Bösch*, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher

Tales from the Dark Side: Privacy DarkStrategies and Privacy Dark PatternsAbstract: Privacy strategies and privacy patterns arefundamental concepts of the privacy-by-design engineer-ing approach. While they support a privacy-aware de-velopment process for IT systems, the concepts used bymalicious, privacy-threatening parties are generally lessunderstood and known. We argue that understandingthe “dark side”, namely how personal data is abused,is of equal importance. In this paper, we introduce theconcept of privacy dark strategies and privacy dark pat-terns and present a framework that collects, documents,and analyzes such malicious concepts. In addition, weinvestigate from a psychological perspective why privacydark strategies are effective. The resulting frameworkallows for a better understanding of these dark con-cepts, fosters awareness, and supports the developmentof countermeasures. We aim to contribute to an eas-ier detection and successive removal of such approachesfrom the Internet to the benefit of its users.

Keywords: Privacy, Patterns

DOI 10.1515/popets-2016-0038Received 2016-02-29; revised 2016-06-02; accepted 2016-06-02.

1 Motivation and IntroductionOver the last years, privacy research has primarily fo-cused on (i) a better conceptual understanding of pri-vacy, (ii) approaches for improving and enhancing pri-vacy protection, as well as (iii) technical mechanisms forimplementing these approaches.

*Corresponding Author: Christoph Bösch: Insti-tute of Distributed Systems, Ulm University, E-mail:christoph.boesch@uni-ulm.deBenjamin Erb: Institute of Distributed Systems, Ulm Uni-versity, E-mail: benjamin.erb@uni-ulm.deFrank Kargl: Institute of Distributed Systems, Ulm Univer-sity, E-mail: frank.kargl@uni-ulm.deHenning Kopp: Institute of Distributed Systems, Ulm Uni-versity, E-mail: henning.kopp@uni-ulm.deStefan Pfattheicher: Department of Social Psychology, UlmUniversity, E-mail: stefan.pfattheicher@uni-ulm.de

However, online service providers have become moreand more sophisticated in deceiving users to hand overtheir personal information. Up until now, privacy re-search has not studied this development.

An example for this development is the Tripadvi-sor mobile app (depicted in Figure 1), which is a reviewplatform for travel-related content. At first glance, thestarting page asks the user to log in with a personalGoogle+, Facebook, or email account. Taking a closerlook, one notices that a third option is given that offersthe creation of a Tripadvisor account. Furthermore, a“Skip”-button is hidden in the upper right corner, whichwhich skips the login process entirely. When signing inwith Facebook, Tripadvisor wants to gain access to thefriend list, photos, likes, and other information (cf. Fig-ure 1b). This is unnecessary for the main features of theservice.

Skipping the login process shows the user some fea-tures which are available only after signing in (cf. Fig-ure 1c). In addition, the “Later”-button, which finallyleads to the app, is located on the left side. Placed on theright side is a “Sign in”-button which leads back to thestarting page. This influencing towards logging in viaFacebook/Google+ or creating a personal account givesTripadvisor access to personal information. Figure 1 il-lustrates general reusable strategies for deceiving usersto share more of their personal information.

In this paper, we deliberately change sides and ex-plore the techniques used by the “bad guys” to col-lect privacy-sensitive data more efficiently. Similar tothe collection of well-established privacy solutions (so-called privacy patterns [14, 24]) as part of the privacy-by-design strategy, we identify and collect malicious pat-terns that intentionally weaken or exploit the privacy ofusers, often by making them disclose personal data orconsent against their real interest.

This approach may initially seem suspicious, as itcould provide guidance for malign stakeholders such asdata-driven companies or criminals. However, we believethat this shift in perspective is helpful and necessary forprivacy research, as it introduces several benefits: (i)A detailed analysis and documentation of privacy darkpatterns allows for a better understanding of the under-lying concepts and mechanisms threatening the users’

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 238

(a) The starting page. (b) Log-in with Facebook. (c) Skipped sign-in process.

Fig. 1. Screenshots of the Tripadvisor mobile app. (a) shows the starting page. Note the small “Skip” button in the upper right corner.(b) shows the requested personal information when logging in with Facebook. Some of the information is unnecessary for providing theservice. (c) shows what happens after skipping the sign-in process.

privacy. (ii) A collection of privacy dark patterns fos-ters awareness and makes it easier to identify such ma-licious patterns in the wild. (iii) Furthermore, the doc-umentation of a privacy dark pattern can be used as astarting point for the development of countermeasures,which disarm the pattern and re-establish privacy. Thediscussion is similar to IT security, where investigationand publication of vulnerabilities proved to be key foractually enhancing the security level in real systems.

1.1 Introduction to Patterns

In many disciplines recurring problems have been ad-dressed over and over again, yielding similar and recur-ring solutions. The idea of a pattern is to capture aninstance of a problem and a corresponding solution, ab-stract it from a specific use case, and shape it in a moregeneric way, so that it can be applied and reused invarious matching scenarios.

Patterns originate from the realm of architecture,where Alexander et al. [5] released a seminal book onarchitectural design patterns in 1977. In this book, theauthors compiled a list of archetypal designs for build-ings and cities which were presented as reusable solu-tions for other architects. Interestingly, Alexander et al.already came up with patterns for privacy. For instance,their Intimacy Gradient pattern postulates a placement

of chambers in such a way that a further distance fromthe building’s entrance allows for increased intimacy.

In 1987, the idea of patterns was readopted by Kentand Cunningham [10] and introduced into the realm ofcomputer science and software development. The Port-land Pattern Repository of Kent and Cunningham col-lected patterns for programmers using object-orientedprogramming languages.1 The idea of using patterns insoftware design gained wide acceptance in 1994, whenthe so-called Gang of Four released their well-knownbook on design patterns for reusable object-orientedsoftware [19]. Since then, the usage of patterns hasspread to various different branches of computer scienceand software engineering, including distributed architec-tures [18, 25], user interface design [46], IT security [41],and privacy [14, 22, 40, 42].

The success of patterns in software engineering hasentailed novel classes of patterns with different seman-tics, namely anti patterns [14] and dark patterns [11].Traditional design patterns capture a reasonable andestablished solution. In contrast, an anti pattern docu-ments a solution approach that should be avoided, be-cause it has been proven to represent a bad practice.

1 Historical side note: the online version of the pattern reposi-tory, WikiWikiWeb (http://c2.com/cgi/wiki), became the first-ever wiki on the World Wide Web

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 239

Hence, anti patterns raise awareness of sub-par solu-tions and advocate against their usage.

Anti patterns often target solutions that may seemobvious to the system developer at a first glance, but in-clude a number of less obvious negative implications andconsequences. Even established design patterns some-times become obsolete and are downgraded to anti pat-terns due to new considerations. For instance, the Gangof Four suggested a pattern for restricting the instan-tiation of a class to a single instance, the so-calledSingleton pattern. Only after concurrent programmingand multi-core architectures became more widespread,shortcomings of this pattern eventually became appar-ent. Today, the Singleton pattern is widely consideredan anti pattern.

The term dark pattern was first used by Brignull,who collected malicious user interface patterns [11] forbetter awareness. A UI dark pattern tricks users intoperforming unintended and unwanted actions, based ona misleading interface design. More generally speaking,a dark pattern describes an established solution for ex-ploiting and deceiving users in a generic form.

In summary, anti patterns collect the Don’ts forgood intentions and dark patterns collect potential Dosfor malicious intents. In this paper, we present a firstbroad discussion on dark patterns in the field of privacy.

1.2 Methodology

To suggest a framework for the collection of privacy darkpatterns and to compile a list of such patterns, we con-sider the problem from three different angles as part ofa holistic approach.

First, we survey existing literature on privacystrategies and privacy patterns. We then reverse pri-vacy strategies and adapt some of these ideas and extendthem, so that they become malicious patterns. Beyondthis, we have identified new types of patterns. Second,we include a psychological point of view on malevolentprivacy concepts. This perspective takes into accounthuman information processing, social cognition and mo-tivation, as well as exploitable basic human needs. Onthis basis we are able to deduce additional approacheson how to reduce the power of privacy dark strategies.Third, we identify and analyze real-world examples ofmalicious privacy mechanisms as found on websites andin mobile applications.

Next, we integrate these findings on privacy darkpatterns into a unified framework, which introduces a

general terminology for privacy dark patterns and es-tablishes a template for documenting privacy dark pat-terns. Our framework suggests a list of malicious privacystrategies and psychological aspects for categorizing pri-vacy dark patterns. Based on the pattern template ofour framework, we discuss common privacy dark pat-terns that we extracted from real-world occurrences.

1.3 Contribution

Our contribution can be summarized as follows:1. We introduce the concept of privacy dark strategies

and privacy dark patterns.2. We present a framework for privacy dark patterns

that takes into account traditional privacy patterns,empirical evidence of malign patterns, underlyingmalicious strategies, and their psychological back-ground. The resulting framework provides a tem-plate for documenting and collecting arbitrary pri-vacy dark patterns.

3. We provide an initial set of exemplary dark patternsthat we encountered in the wild.

4. We launched the website dark.privacypatterns.eu asan online collection for privacy dark patterns. Beinga collaborative resource, we invite the community tosubmit more patterns and help to raise awareness.

2 On Privacy Strategies andPrivacy Patterns

In this section, we introduce privacy patterns and cor-responding privacy strategies, based on their historicaldevelopment.

Until the mid-1990s, privacy was rarely considereda relevant feature of IT systems. Even if it was, the inte-gration of privacy-preserving mechanisms was often con-ducted a posteriori, as an additional requirement lateradded to the system. The notion of “privacy as an af-terthought” contradicted the cross-sectional property ofprivacy as part of an IT system and often yielded ex-tensive or insufficient changes to the system.

To overcome these deficits, a joint team of the Infor-mation and Privacy Commissioner of Ontario, Canada;the Dutch Data Protection Authority; and the Nether-lands Organisation for Applied Scientific Research ad-vocated a more integral approach that included privacyconsiderations into the overall development cycle [27]. In1995, they introduced the so-called Privacy by Design

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 240

approach,2 which is postulated by the following sevenfoundational principles:1. Proactive not reactive2. Privacy as the default setting3. Privacy embedded into design4. Full functionality5. End-to-end security6. Visibility and transparency7. Respect for user privacy

These principles have been a major milestone for the de-sign of privacy-preserving systems as they provide gen-eral guidance. For that reason, these concepts are partof several privacy legislations today.

One frequent criticism regarding Privacy by Designand its seven principles is that they are too unspecificto be directly applied to a development process. Theprinciples neither provide concrete advice, nor do theyaddress the varying needs of specific domains, such asthe Internet of Things, User Interface Design, or Car-2-Car communication. A system designer is thus stillrequired to have a thorough understanding of privacyand the forces involved, to design a privacy friendly sys-tem. Clearly, more guidance and a more methodologicalapproach is required to establish a privacy engineeringprocess as, for example, worked on by the PRIPAREproject [38].

One element of this privacy engineering approachare so-called Privacy Patterns. The main idea of pri-vacy patterns is to improve the drawbacks of the Pri-vacy by Design principles, i.e., that they are not action-able [14, 21, 48]. Privacy patterns are defined as reusablesolutions for commonly occurring problems in the realmof privacy. Essentially, they are patterns for achievingor improving privacy. Privacy patterns provide guidancefor engineering and development, and target the needsof specific domains, such as backend implementations oruser interface design. By providing a well-structured de-scription of a problem and its solution using a standard-ized template, patterns can easily be looked up and ap-plied. Since these patterns include references to specificuse-cases and possibly implementations, engineers willdirectly find the resources needed to implement them intheir own context.

One well-known example of a privacy pattern thatcan be implemented in multiple domains is the strip-ping of metadata that is not necessary for the function-ality of the service. This procedure increases privacy,

2 https://privacybydesign.ca/

since metadata often includes personally identifiable in-formation. Further, this solution is reusable, since it isnot bound to a specific instance of a problem. Thus,stripping of metadata constitutes a privacy pattern thatcan be applied, e.g., to a website managing digital pho-tographs.

A single privacy pattern addresses a problem witha limited scope. Multiple related and complementingpatterns can then be compiled into a pattern catalog.Similar to the well-known design pattern catalogs fromsoftware engineering, a privacy pattern catalog collectsa number of relevant problems and suitable solutionsthat can be applied during a privacy-aware developmentphase.

There are multiple collections of privacy patternsfrom academic research [14, 22, 40, 42] as well as onlinerepositories3 that are more accessible for practitioners.

In a typical system development process, privacypatterns are applied during the stages of design and im-plementation. However, in many scenarios, privacy as-pects represent fundamental system requirements thathave to be considered from the very beginning. Thequestion is, whether more general architectural build-ing blocks exist that can be applied at an even earlierstage, i.e., during requirement analysis and architecturaldesign. Note that this notion is a natural continuationof the Privacy by Design philosophy—to include privacyconsiderations into the entire development process.

These general architectural building blocks areknown as Privacy Design Strategies. According to Hoep-man [24], a privacy design strategy is on a more generallevel than a privacy pattern and “describes a funda-mental approach to achieve a certain design goal. It hascertain properties that allow it to be distinguished fromother (fundamental) approaches that achieve the samegoal.”

Later in the development process, a privacy designstrategy can be refined with privacy patterns imple-menting one or more strategies. Thus, privacy designstrategies provide a classification of privacy patterns.When system designers search for a privacy pattern ina collection, they are only interested in the ones imple-menting their chosen privacy strategy.

Hoepman [24] defines the following eight privacy de-sign strategies.Minimize: Data minimization is a strategy which in-

sists that the amount of personal information thatis processed should be minimal. Data that is not

3 https://privacypatterns.eu, http://privacypatterns.org/

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 241

needed for the original purpose should not be col-lected.

Hide: Hide takes place after data collection. WhereasMinimize forbids the collection of needless informa-tion, Hide suggests that any personal data that isprocessed should be hidden from plain view.

Separate: The approach of the privacy strategy Sep-arate is to process any personal information in adistributed fashion if possible. Thus, interrelation-ships between personal data vanish in contrast to acentralized processing.

Aggregate: When implementing Aggregate, per-sonal information is processed at a high level of ag-gregation. This level should only be so high as toremain useful, however. Details that are not neededfor the functionality of the service vanish. This pro-cess could include statistical aggregation such thatthe details of identities are blurred.

Inform: The privacy strategy Inform states that datasubjects should be adequately informed wheneverpersonal information is processed.

Control: A common requirement of software systemsis that data subjects should be in control of the pro-cessing of their personal information. Whenever thisis ensured, we are dealing with the privacy strategyControl. Hoepman states that he is not aware ofany patterns implementing this strategy.

Enforce: Enforce states that a privacy policy thatis compatible with legal requirements should be inplace and should be enforced.

Demonstrate: The privacy strategy Demonstratedemands that data controllers are able to demon-strate compliance with their privacy policy and anyapplicable legal requirements. A good example fora pattern implementing this strategy is the use ofaudits.

In the following sections, privacy design strategies serveas the starting point for our analysis of malicious darkstrategies that harm privacy. For defining and docu-menting malicious patterns, we adapt the idea of privacypatterns and transform it into privacy dark patterns.

3 The Dark SideThe triad of general privacy strategies for high-level pri-vacy requirements, privacy patterns for privacy-awaredesign processes, and privacy-enhancing technologies

for system implementations is commonly acknowledgedwhen building privacy-friendly IT systems.

However, there are other parties that have differentagendas when building IT systems. Instead of privacy-friendly solutions, they aim for systems that purpose-fully and intentionally exploit their users’ privacy—forinstance motivated by criminal reasons or financially ex-ploitable business strategies.

For the development of our framework, we re-verse the evolution of privacy strategies and patterns:First, we define dark strategies as the high-level goalsthat these parties follow in order to exploit privacy.Next, we derive suitable dark patterns that implementthese strategies. We then complement our framework byadding a psychological perspective on how the strate-gies generally achieve their deceptive and manipulativegoals. Note that we do not include a counterpart toprivacy-enhancing technologies as part of our frame-work.

As already clarified in the introduction, the result-ing framework is neither intended nor structured asa construction kit for malicious parties. Instead, theframework can be used by privacy researchers and prac-titioners for detecting, recognizing, analyzing, and doc-umenting malicious strategies and patterns.

When used top-down, the framework supports a pri-vacy analysis of IT systems by raising awareness formalicious strategies and by uncovering correspondingmechanisms. Bottom-up, the framework helps to iden-tify malicious patterns, reveals underlying strategies,and provides pointers for the development of concretecountermeasures.

3.1 Privacy Dark Strategies

We now develop a categorization of privacy dark pat-terns, analogously to Hoepman’s privacy design strate-gies [24]. As privacy design strategies can be used to cat-egorize privacy patterns by their fundamental approach,the same holds for privacy dark strategies. Hoepmanidentified eight privacy strategies, namely Minimize,Hide, Separate, Aggregate, Inform, Control, En-force, and Demonstrate. Based on these strategies,we identify the following privacy dark strategies: Max-imize, Publish, Centralize, Preserve, Obscure,Deny, Violate, and Fake as shown in Table 1. Theseare used for our categorization of privacy dark patternsin Section 5.

The privacy design strategy Minimize, for exam-ple, demands the amount of processed data to be re-

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 242

Table 1. Privacy Strategies vs. Dark Strategies.

StrategiesHoepman Dark Strategies

Minimize MaximizeHide PublishSeparate

VSCentralize

Aggregate PreserveInform ObscureControl DenyEnforce ViolateDemonstrate Fake

stricted to the minimal amount possible. The corre-sponding dark strategy Maximize would collect, store,and process as much data as possible, leading to a lossof privacy. The system designer does not act out of puremaliciousness but to gain an advantage over a systemwith the same functionality but with stronger privacyprotection. Specifically, by receiving additional personaldata which can, e.g., be sold or used for personalized ad-vertisements.

In the following we detail the eight privacy darkstrategies we have developed.

Maximize. The goal of the dark strategy Maximize isto collect an inappropriate amount of data. More pre-cisely Maximize means that. . .

The amount of personal data that is collected,stored, or processed is significantly higher than whatis actually needed for the task.

Examples would be extensive sign-up forms with fieldsthat are not needed for the functionality of the service.Often those unneeded fields are mandatory, maximizingthe collection of personal data. Another example of aMaximize strategy are bad default privacy settings orthe necessity to set up an account for the usage of aservice, especially if the account is not needed for thefunctionality of the service.

Publish. The dark strategy Publish can be character-ized by the requirement that. . .

Personal data (not intended to be public) is not hid-den from plain view.

This means that often no mechanism is in place to hidepersonal data from unauthorized access, such as en-cryption or access control. The personal data lies in theopen for everyone to see. Social networks often employ

this dark strategy to encourage the sharing of personaldata and thus the use of their platform. This strategysatisfies a person’s need to belong as will be explainedin section 4.

Centralize. Centralize is the dark strategy associ-ated to the privacy strategy Separate, which mandatesthat personal data should be processed in a distributedway. Centralize, in contrast, enforces that. . .

Personal data is collected, stored, or processed at acentral entity.

This strategy preserves the links between the differentusers and thus allows for a more complete picture oftheir habits and their usage of the service.

Advertising networks employ this strategy heavilyby sharing pseudonymous user IDs, a practice known ascookie syncing [1]. Another common occurrence of thisprivacy dark strategy is the practice of flash cookies,which are cookies that are stored centrally by the flashplug-in on the file system and are thus not restricted toa specific web browser.

Preserve. The dark strategy Preserve requiresthat. . .

Interrelationships between different data itemsshould not be affected by processing.

They should rather be preserved in their original statefor analysis instead of storing them in a processed form,e.g., aggregation. It is not necessary to know the type ofanalysis in advance. A prominent example is telecom-munications data retention because traffic analysis canrecover the relationships between persons.

Obscure. In the dark strategy Obscure. . .

It is hard or even impossible for data subjects tolearn how their personal data is collected, stored,and processed.

Users should be unable to inform themselves about whathappens to their disclosed data. This can be achieved inthe form of a privacy policy with many technical terms,which are difficult to understand for the average user.User Interfaces could be designed to mislead the user,leading to decisions contradicting the user’s original in-tent. The EFF called this particular mechanism “pri-vacy zuckering” [28].

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 243

Deny. Patterns making use of the dark strategy Denymake a data subject lose control of their personal data.The term Deny is due to a denial of control.

Data subjects are denied control over their data.

With this dark strategy, a service provider can pre-vent users from taking actions that oppose that serviceprovider’s interest. An example is to not provide thefunctionality for deleting an account. Another exampleis the nonexistence of options to control sharing of infor-mation. Until recently this was the case in WhatsApp,where the online status was automatically shared witheveryone who subscribed to that phone number, whichhas a big impact on the privacy of users [12].

Violate. The strategy Violate occurs if. . .

A privacy policy presented to the user is intention-ally violated.

A privacy policy is in place, shown to the user butintentionally not kept. The users are unaware of theviolation; thus, this does not impact the trust put intothat service if such violations are not revealed. It is hardto find concrete examples and patterns implementingthis strategy since using this strategy is against the lawand not publicly admitted by companies.

Fake. The privacy dark strategy Fake means that. . .

An entity collecting, storing, or processing personaldata claims to implement strong privacy protectionbut in fact only pretends to.

An example of this strategy are self-designed padlockicons or privacy seals, which make the user feel securebut do not have any meaning. Another example arewrong and unsubstantial claims such as an unrealisticclaim on the key-size of ciphers or marketing terms like“military grade encryption”.

SynthesisOur eight Privacy Dark Strategies can be summarizedas follows:– Maximize: The amount of personal data that is col-

lected, stored, or processed is significantly higherthan what is actually needed for the task.

– Publish: Personal data is published.

– Centralize: Personal data is collected, stored, orprocessed at a central entity.

– Preserve: Interrelationships between differentdata items should not be affected by processing.

– Obscure: It is hard or even impossible for data sub-jects to learn how their personal data is collected,stored, and processed.

– Deny: Data subjects are denied control over theirdata.

– Violate: A privacy policy presented to the user isintentionally violated.

– Fake: An entity collecting, storing, or processingpersonal data claims to implement strong privacyprotection but in fact only pretends to.

3.2 Privacy Dark Patterns

After our exploration of privacy dark strategies we willnow define the concept of a privacy dark pattern. Asmentioned in Section 2, a pattern describes a generic,reusable building block to solve a recurring problem andhence to document best practices. They can be collectedin special catalogs and allow for easy replication. Pat-terns fulfill the role of a common language to allow sys-tem developers and privacy engineers to communicatemore efficiently.

We argue that common building blocks that areused by service providers to deceive and mislead theirusers exist. Some service providers use recurring pat-terns to increase the collection of personal data fromtheir users. Sometimes these building blocks are usedunintentionally, simply constituting usage of privacyanti patterns, but without any malicious intent. How-ever, we claim that there are building blocks which areused on purpose, thereby yielding an advantage to theservice provider. We call these building blocks privacydark patterns.

Analogously to privacy patterns, privacy dark pat-terns can be collected in special repositories to facilitateeasy access and retrievability for users and to developcountermeasures. Patterns are usually documented ina formalized template to enable system developers toeasily reference and use them. Common fields in such atemplate include the name of the pattern, the problemthe pattern is solving and references to related patterns.

However, current templates for design and privacypatterns are not suitable for documenting privacy darkpatterns due to the following reasons:1. Privacy patterns and privacy dark patterns have

a different intent regarding their documentation.

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 244

Each privacy pattern solves a specific problem,which is often mentioned as a separate field in thetemplate. Privacy patterns are documented to becopied and used. The purpose of documenting pri-vacy dark patterns on the other hand is to createand enhance awareness about common anti-privacytechniques, since they do not solve an engineeringproblem. Thus, a problem-centric description is outof place.

2. The target group of privacy patterns are systemdesigners whereas privacy dark patterns can targetnon-technical end-users to educate them about thestrategies that are used to deceive them.

Thus, we need a different template to document privacydark patterns.

Our Privacy Dark Pattern Template

We have developed a new template, specifically targetedtowards privacy dark patterns, which we explain in de-tail in the following.Name/Aliases: This field describes the name under

which the privacy dark pattern is known. The nameshould be concise and capture the essence of thepattern.

Summary: A short summary to describe the patternis necessary to provide an overview of the patternand for quick reference.

Context: Context describes the scenario in which thepattern appears. e.g., online social networks or on-line shops.

Effect: This section explains the effects and conse-quences of the pattern. This should be describedwith sufficient granularity such that it is not toogeneral.

Description: In this part of the template, the privacydark pattern is described in detail. Technical lan-guage can be used if not avoidable, but it shouldbe remembered that the main target group of thepattern are the end-users of the system in which theprivacy dark pattern is applied.

Countermeasures: The countermeasures describe be-haviors and tools a user can implement to negate theeffects of the privacy dark pattern. These are strate-gies to help the “victims” of the pattern regain ormaintain their privacy. This includes procedures toavoid the effects of the pattern, as well as add-ons toexisting programs, e.g., web browsers, which preventthe end-user from being deceived by the pattern.

Examples/Known Uses: In this section, implemen-tations using the dark pattern are described. Serviceproviders applying the privacy dark pattern belonginto this field. Screenshots of usage of the dark pat-tern can be provided where appropriate.

Related Patterns: If related privacy dark patternsexist they are referenced here.

Psychological Aspects: This field describes the psy-chological mechanisms that make the pattern effec-tively influence the users in their behavior.

Strategies: In this part of the documentation of a pri-vacy dark pattern, the used dark strategy is pro-vided. These are the dark strategies explained inSection 3.1.

This template can be used to systematically documentdifferent privacy dark patterns in a repository. We makeuse of this template later in Section 5.

4 Psychological AspectsIn the following, we address the question why privacydark patterns do actually work. One can reasonably as-sume that there is, at least to some degree, awarenessamong a majority of users that privacy dark strategiesexist and some service providers have strong incentivesto violate the privacy of their users. It is similarly likelythat users notice, at least sometimes, when they are be-ing targeted by privacy dark strategies. Nevertheless,privacy dark strategies still work, as indicated by theirfrequent occurrence. This somewhat paradoxical situa-tion can be explained by adopting a psychological per-spective on privacy dark strategies.

Essentially, privacy dark strategies often work wellbecause they take advantage of the psychological consti-tution of human beings. In this regard, we focus on theways in which humans think and reason, i.e., humans’cognitive information processing.

There is widespread agreement in the field of psy-chological research that two different cognitive systemsunderlie thinking and reasoning processes [29, 43, 44].For instance, when creating a new account on a web-site, a users are often asked to agree to a list of generalterms and conditions. Most likely, they will not readthe page filled with these terms and conditions, but willagree to them quickly, intuitively, and automatically.This is an example of a System 1 thinking process; ittakes place automatically, unconsciously, and with littleeffort [29, 43, 44].

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 245

Instead of agreeing to general terms and conditionsquickly and automatically, one can take the time andmake the effort to carefully read the information pro-vided. Afterwards, one deliberatively weighs the prosand cons and decides whether to agree to the condi-tions or not. This is an example of a System 2 thinkingprocess; it takes place in a controlled, conscious, andeffortful way. Behavior based on System 2 thinking isdriven by a deliberative, effortful decision-making pro-cess, resulting in the relatively slow execution of behav-ior [29, 43, 44].

General terms and conditions are often not read,and agreement is typically made automatically andquickly, i.e., System 1 operates. There is thus an op-portunity to fill general terms and conditions with darkingredients. These in turn are not consciously noticedwhen users are in System 1 mode, as illustrated in theexample. In general, we postulate that privacy darkstrategies work well when individuals process informa-tion using System 1 thinking. When (dark) informationis processed quickly, without much effort, and automat-ically, it seems likely that privacy dark strategies canunleash their full impact. In other words, in System 1mode, subjects are likely to be less conscious of privacydark patterns being at work and unable to delibera-tively act against them. On the other hand, recognizingprivacy dark strategies and taking action against themrequires System 2 processing.

Past research in fact shows the importance of cog-nitive information processing for privacy issues (e.g.,[8, 32, 34]). Knijnenburg and colleagues [31], for in-stance, document that people automatically providepersonal information on website forms when an auto-completion feature fills out forms by default with pre-viously stored values. Reducing the impact of this au-tomatic (System 1 based) default completion by givingusers control over which forms are filled out reduces theamount of personal information provided.

A number of conditions determine whether humansrely on System 1 thinking processes and System 2 think-ing processes are inhibited. There are two central as-pects to consider [16, 39]. Humans engage in System 1processing whenever they (a) have little motivation tothink and reason in an effortful way or (b) have no op-portunity to do so because they lack the required knowl-edge, ability, or time. Users, for instance, often have nomotivation to read general terms and conditions. In in-stances where they are motivated, they often do nothave the opportunity to use System 2 thinking becausethe language used in general terms and conditions often

is too complicated and subjects are unable to interpretthis information [35].

4.1 Prompting System 1 Thinking

As argued above, privacy dark strategies are typicallyaccompanied by System 1 thinking processes, while Sys-tem 2 thinking processes are often not possible, as shownin the following analysis. Regarding the dark strategyMaximize, the amount of data that is processed is sig-nificantly higher than the data that is really needed forthe task. Subjects need high motivation to resist exces-sive data collection. Additionally, although some usersmight have high motivation, they need specific knowl-edge and abilities to offer any resistance. However, someservice providers use mandatory form fields for user reg-istration, which renders the knowledge to circumventthe dark strategy useless if one wants to utilize the ser-vice. Thus, users often stay in System 1 mode and allowMaximize to operate.

When personal data is not hidden from plain view(Publish), users need to be motivated and able tochange settings. Users might lack the necessary moti-vation and ability to do so; thus, remaining in System 1processing when it comes to, for instance, privacy set-tings.

Working against the dark strategies of centralizingpersonal data (Centralize) and of providers that inter-relate data items (Preserve) requires particularly highmotivation as well as extensive knowledge of and theability to understand these strategies. It is reasonableto assume that the typical user often does not have theknowledge and ability to precisely understand the darkstrategies of Centralize and Preserve and to workagainst them (e.g., taking action against data preserva-tion). Thus, users often cannot engage in the delibera-tive processing that might lead to behavior that chal-lenges these two dark strategies.

The dark strategy Obscure reflects the idea that itis difficult for users to learn about what happens to theirpersonal data. This strategy implies that users must behighly motivated and able to acquire information abouthow their personal data is used and stored. Again, thisrequirement inhibits users from engaging in deliberativeprocessing.

Analogously, when users’ control of data is denied(Deny) they must be highly motivated and able to workagainst this strategy. Deny makes it even more diffi-cult for users to notice the violation of privacy policiesand legal requirements (Violate). Here, high motiva-

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 246

tion and ability is needed to enable users to notice andto work against this dark strategy.

When certificates or other information is faked(Fake), users need to be motivated to search for thisinformation. Additionally, they need the ability to judgewhether information has been faked or not. If motiva-tion and ability is not present, subjects will process(fake) information using System 1 thinking and willlikely not notice the privacy dark strategy.

To sum up, it is evident that privacy dark strate-gies work, because users often do not have the mo-tivation or opportunity to resist them. As such, Sys-tem 2 thinking processes are often absent, while Sys-tem 1 thinking does accompany the use of privacy darkstrategies. Building on these considerations, one can de-duce suggestions on how to reduce the power of privacydark strategies. Specifically, we argue for attempts tostrengthen System 2 thinking processes by increasingmotivation (e.g., through emphasizing the negative im-pact of privacy dark strategies) and opportunities forresistance (e.g., by increasing knowledge about privacydark strategies as advocated by this paper, or by imple-menting tools that reduce automatic provision of privateinformation [31]).

4.2 Humans’ Fundamental Need toBelong

Beyond the idea that human information processing isinvolved in the functioning of privacy dark strategies,humans’ fundamental needs also contribute to the effec-tiveness of some privacy dark strategies. Humans pos-sess basic needs, e.g., safety and security needs, concernsabout physical well-being, the need for self-esteem, andthe need to belong to significant others [23]. We iden-tified the need to belong as particularly important forwhy some privacy dark strategies work well. The argu-ment that is put forward states that individuals’ needto belong forces people to disregard privacy issues.

The need to belong reflects humans’ desire to be anaccepted member of a group. Psychological experiments(e.g., Williams et al. [49]) show that social exclusion ofa subject, even by unknown other subjects in a sim-ple ball game played on the Internet, reduced subjects’well-being, their belief in a meaningful existence, andtheir self-esteem. People’s need to belong manifests asa concern for being liked and admired by others, as isevident in social networks [20]. The need to belong mo-tivates people to accumulate social capital [9], i.e., toestablish relationships with other people (e.g., in social

Fig. 2. Dialog of the Facebook mobile website when deactivatingthe account. The page shows profile pictures of contacts the userhas recently interacted with and states that they will miss theuser when deactivating the account. Facebook targets the user’sneed to belong and provokes a reconsideration.

networks) that serve as personal resources for individu-als’ well-being and functioning [7, 15, 26].

Although important for human beings [9], the needto belong might counteract privacy concerns. For exam-ple, when personal data is not hidden from plain view(Publish), it can create a possibility of being liked andadmired by others, which can fulfill one’s need to belong(cf. Nadkarni and Hofmann [37]). This may lead to a re-duced level of privacy at the same time. Furthermore,it is hard for subjects to learn about what happens tothe personal data (Obscure) they share based on theirneed to belong.

Service providers might further Maximize theamount of data based on subjects’ need to belong togain information about their users, specifically abouttheir social capital [15]. This information is then usedto again target subjects’ need to belong, for instancewhen a user wants to unsubscribe. Facebook, for exam-ple, writes “Your [number] friends will no longer be ableto keep in touch with you.”, and “[Name] will miss you”

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 247

(status January 16, 2016). As shown in Figure 2, users’motivation to unsubscribe is challenged by activatingtheir need to belong and the presentation of users’ socialcapital they would lose once they unsubscribe [7, 26].

In sum, people provide and share private informa-tion based on their need to belong. Therefore, the needto belong may run counter to high privacy standards.

4.3 Specific Mechanisms

In summary, privacy dark strategies often work well be-cause they take advantage of human beings’ psycholog-ical constitution. We argue that System 1 thinking andthe need to belong are so fundamental for malicious pri-vacy mechanisms to work, that both aspects representthe basis of psychological considerations in our frame-work. Furthermore, we believe that both aspects arehelpful for contributors when briefly assessing potentialprivacy dark patterns and their psychological mechan-ics.

The discussion whether a pattern is to be regardedas a dark pattern can then easily integrate a perspec-tive of the users. This psychological perspective com-plements the assessments of actual impacts of the pat-tern and suspected motives of the service providers.This is important in order to differentiate actual privacydark patterns with malicious intent from other forms ofpoorly implemented or unintended features regardingprivacy.

Apart from the thinking and reasoning processesand the need to belong mentioned before, arbitrary pat-terns may exploit more specific psychological mecha-nisms which build upon these fundamental aspects. Inthe following, we introduce some of these mechanismsand indicate their usage for privacy dark patterns.

First, we focus on nudging, a concept for influenc-ing decision making based on positive reinforcement andnon-forced compliance [45]. Nudging has already beenapplied to decision making in the domain of privacy pro-tection [3]. For instance, regular nudges that provide auser with information about data collection of smart-phone applications have shown to increase awarenessand motivate users to reassess the applications’ permis-sions [6]. When the good intents of privacy nudging arereplaced with malicious intents, the concept turns intoa latent manipulation technique for non-forced compli-ance with weakened privacy. The dark counterpart pro-vides choice architectures facilitating decisions that arenegative to the user’s privacy. For instance, the startingscreen in Figure 1 does not force an account creation

and it provides a skip option. Still, the form design la-tently manipulates the user by encouraging the creationof a user account.

A stronger form of manipulation is achieved by ap-plying traditional persuasion techniques [13]. For in-stance, the so-called “door in the face” technique takesadvantage of the principle of reciprocity. In this tech-nique, the refusal of a large initial request increases thelikelihood of agreement to a second, smaller request.This technique has already been studied in the contextof private information disclosure [4] and privacy usersettings [30]. Applied to privacy dark strategies, a ser-vice provider might intentionally ask users for dispro-portionate amounts of personal data. By providing anoption to skip the first form and then only asking fora very limited set of personal data in the second form(e.g., mail address only), users may be more willing tocomply and to provide that information after all.

Closely related to the two cognitive systems, heuris-tics and biases provide decision guidance in case of un-certainty [47]. Although there are a lot of heuristicsand cognitive biases related to decision making [29] thatcould be exploited by dark privacy patterns, we will onlyintroduce an exemplary bias that we later use in one ofour example patterns: Hyperbolic discounting [33] is abias causing humans to inconsistently valuate rewardsover time. Also known as present bias, this bias trickshumans into favoring a present reward over a similar re-ward at a later point in time. In terms of privacy, manyusers tend to focus on the instant gratification of animmediate reward, when they are forced to provide per-sonal data to use a service. At the same time, the usersdiscount the ramifications of privacy disclosures in thefuture [2].

Cognitive dissonance [17] is a state of discomfortcaused by contradictory beliefs and actions. Accordingto the theory of cognitive dissonance, the experience ofinconsistency triggers a reduction of dissonance and apotential modification of the conflicting cognition. Interms of privacy dark patterns, this process can be ex-ploited by inconspicuously providing justification argu-ments for sugarcoating user decisions that have nega-tively affected their privacy. For instance, after askingusers for inappropriate amounts of personal data, a ser-vice provider would later remind the users of the highdata protection standards they comply with. When auser hesitantly provides personal data although they aregenerally very cautious regarding personal information,a state of discomfort may emerge soon after. Such hintsmay then influence the dissonance resolution of the user.

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 248

5 Dark Patterns in the WildThis section introduces patterns of frequent maliciousprivacy behavior. For this purpose, we surveyed popularweb sites and mobile applications and gathered reportsof recent privacy incidents. Next, we analyzed the un-derlying concepts and mechanisms regarding maliciouseffects on privacy, assessed their impacts, and estimatedthe intentionality of the service providers. Based on ourframework, we then extracted a number of common darkprivacy patterns and described them using our patterntemplate. The resulting list is not exhaustive, but illus-trates the idea of privacy dark patterns based on exem-plary sightings in the wild.

Of course we cannot clearly determine whether theservice providers mentioned as examples in the followingpatterns actually had a malicious intent, and we are notclaiming they did. It is still reasonable to believe thatmany of the companies offering free services and appshave strong motivations to gather as much data fromtheir customers as possible and design their mobile webservices and mobile applications on purpose followingsuch privacy dark patterns. In any case, the examplesare helpful to understand the mechanics of the privacydark pattern in question.

Please note that the following patterns are short-ened and use a condensed structure. The extendedversions of the patterns based on our full tem-plate structure are available at our online portaldark.privacypatterns.eu.

5.1 Privacy Zuckering

The term Privacy Zuckering was first introduced byTim Jones in an EFF article [28] for “deliberately con-fusing jargon and user-interfaces”, and was later usedon darkpatterns.org for a UI dark pattern. For our cat-alog, we generalize the idea and present it as a universalprivacy dark pattern.

Name/Aliases: Privacy ZuckeringContext: The access and usage of personal data isoften governed by user-specific, modifiable privacy set-tings. By doing this, users can choose privacy settingsthat reflect their own privacy requirements.Description: A service provider allows users to changetheir privacy settings. However, the settings are unnec-essary complex, overly fine-grained, or incomprehensi-ble to the user. As a result, the user either gives up, or

makes unintended changes to their privacy settings.Effect: While the service provider will claim that usershave full control over their privacy settings, the pre-sentation, terminology and user experience will highlydiscourage users from making changes. When combinedwith the Bad Defaults pattern, these patterns facilitatethe enforcement of privacy settings suggested by the ser-vice provider. Privacy Zuckering could lead to uninten-tional changes of privacy settings, when the complexityof the settings does not align with the user’s percep-tion, and hence prevents originally intended preferenceadjustments.Countermeasures: When service providers apply Pri-vacy Zuckering, users require help of third parties thatclarify the settings and guide them through the intendedpreferences.Examples/Known Uses: In the past, Facebook hasbeen accused of applying Privacy Zuckering to theirusers’ privacy setting pages, which termed the mech-anism in the first place [11]. For instance, in August2010, an updated privacy settings page of Facebook al-lowed for highly customized settings, but required usersto change dozens of settings on multiple pages to max-imize personal privacy.Related Patterns: When Bad Defaults are in place,Privacy Zuckering prevents changes and increases thenumber of retained default settings.Psychological Aspects: Overly complex settings andinappropriate terminology requires System 2 thinking.When a user is motivated to change their settings, butis overwhelmed at the same time, and hence lacks theopportunity to do so purposefully, the user may ei-ther switch back to System 1 thinking and make vaguechanges, or the user may refrain from doing so at all.Strategies Obscure

5.2 Bad Defaults

Name/Aliases: Bad DefaultsContext: This dark pattern is used mainly on websites,by applications, or in social networks. For Bad Defaultsto have an effect it is often necessary that the systemhas some form of user accounts.Description: When creating an account at a serviceprovider the default options are sometimes chosen badlyin the sense that they ease or encourage the sharing ofpersonal information. Most users will be too busy tolook through all the options and configure their accountproperly. Thus, they often unknowingly share more per-sonal information than they intend to.

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 249

Fig. 3. Facebook default settings from 2010. The graph showswhich information can by default be accessed by You, yourFriends, Friends of Friends (FoF), all Facebook user (FBU), andthe whole Internet (Inet). For the source and more details werefer to http://mattmckeon.com/facebook-privacy/.

Effect: This pattern causes the user to share more in-formation with the system or other users than the userintends to do. This includes but is not limited to whichsites the user visits, parts of his user profile, and his on-line status.Countermeasures: Users need to be educated to de-velop more awareness of bad default settings so thatthey become self-motivated to configure their accountsproperly. However this is hard to achieve.Examples/Known Uses: Facebook Default PrivacySettings (cf. Figure 3).Related Patterns: Privacy Zuckering demotivatesusers from changing the defaults.Psychological Aspects: When users are not aware ofthe defaults that are in effect, a deliberative processingof this information is inhibited.Strategies: Obscure

5.3 Forced Registration

Name/Aliases: Forced Registration

Context: This pattern can be applied in nearly ev-ery service which provides some functionality to users.When the functionality technically requires an account,e.g., in online social networks, this pattern degenerates.In this case we are not speaking of a privacy dark pat-tern anymore since without an account the service can-not be provided in the intended way.Description: A user wants to use some functional-ity of a service which is only accessible after registra-tion. Sometimes this is necessary to use the service in a

meaningful way or prevent misbehavior. But very oftenthis is unnecessary and serves the interest of the serviceprovider by giving him access to (unneeded) personaldata. The personal information collected regularly in-cludes an e-mail address, since this is required for cre-ating the account, but is often augmented by birthdates,home addresses, etc.Effect: The effect of this pattern is that the user isforced to register an account at the service provider,thereby allowing the service provider to track user be-havior on his platform. Additionally the registrationprocess often requires an e-mail address and other per-sonal identifiable information. Since the user does notwant to have an account in the first place, the user isunlikely to configure the settings properly, thereby pos-sibly revealing even more personal information not in-tended for disclosure.Countermeasures: One countermeasure is to create anew account and fill it with random data. Often, one canuse an anonymous one-time e-mail address4 during reg-istration to receive the activation link for the account.

Another countermeasure is provided by the serviceBugMeNot5. They enable users to bypass the forcedregistration by allowing many users to share their ac-count details creating a large anonymity set. A user cantry accounts published at BugMeNot for using the ser-vice. BugMeNot allows users to create new accounts andshare them with other users of BugMeNot. It can evenbe used as a browser extension by some web browsers.Examples/Known Uses: As of Feb. 2016, the popu-lar question-and-answer website Quora.com requires ex-ternal visitors to sign up and log in when opening aquestion page. While the page is rendered initially, it isthen blocked by pop-up modal dialog that forces visi-tors to register, even for one-time, read-only access.Related Patterns:When a user is required to register,an Immortal Account will prevent the later cancellationof the account. Forced accounts can come with Bad De-faults.Psychological Aspects: As the user’s original goalis prevented by the necessary registration, account cre-ation often happens as part of an automatic behaviorfor achieving that goal. This gives the user an instantgratification, and critical and deliberative thoughts areinhibited.Strategies: Maximize

4 e.g., http://10minutemail.com5 http://bugmenot.com/

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 250

5.4 Hidden Legalese Stipulations

Name/Aliases: Hidden Legalese StipulationsContext: This pattern can be used by all systemswhich incorporate a document describing the terms andconditions of using the service.Description: Terms and conditions are mandatory bylaw. Nevertheless, most users do not read them, sincethey are often long and written in a complicated legaljargon. This legal jargon is necessary to provide suc-cinctness and clarity, but is not user-friendly.

The inability of the user to grasp the legal jargonputs him in a vulnerable state, since the policy is legallybinding. If this vulnerability is exploited, the policyturns into an instance of a privacy dark pattern. Ser-vice providers can hide stipulations in the policies whichtarget the privacy of the user. Often the user will notnotice this, not reading the terms and conditions or be-ing unable to understand their implications. Some ser-vice providers state that they will change their policieswithout further notice, preventing the user even furtherfrom learning what happens to his data.Effect: Usage of this pattern leads to the serviceprovider being able to hide his malicious deeds fromthe user without necessarily violating legal regulations.Countermeasures: There are various proposals foreasier communication of legal conditions.

One solution is to make the legal conditionsmachine-readable. This was the approach that P3P, thePlatform for Privacy Preferences Project, followed. P3Pis a standard by the W3C6 for a machine-readable ren-dering of privacy policies. The basic idea is that anXML-file specifying the privacy policy can be retrievedfrom any participating web pages. This policy can auto-matically be checked against the preferences of the userby the browser.

The Privacy Bird7, for example, was a tool whichcould show the P3P description as an icon, namely abird. The color or the bird, i.e., red or green, signified ifthe policy of the site matched the users’ preferences.

The drawback of this approach is, that the serviceprovider needs to provide the machine-readable P3P de-scription. A malicious service provider who wants totrick his users with hidden legal stipulations will ofcourse not provide such a description. Since this coun-termeasure depends on the collaboration with the ser-vice provider it is not effective.

6 https://www.w3.org/P3P/7 http://www.privacybird.org/

Another approach is the one followed by the Termsof Service; Didn’t Read (TOSDR8) webpage. This isa community-driven repository of ratings of privacypolicies. TOSDR is available as a browser add-on andshows the rating of the terms of service of the currentweb page as a small icon. When clicking on the icon onecan see the positive and negative points of the terms ofservice in an easily understandable language.

Examples/Known Uses: In 2000, the then-popularinstant messenger service ICQ introduced a “TermsOf Service — Acceptable Use Policy”9 which grantedthe service operators the copyright on all informationposted by their users. Hidden in this legalese, the oper-ators granted further rights of use “including, but notlimited to, publishing the material or distributing it”.

The British firm GameStation owns the souls of7,500 online shoppers, thanks to an “immortal soulclause”10 in the terms and conditions. This April Fool’sgag reveals the effectiveness of this pattern and showsthat companies can hide everything in their online termsand conditions. Please note that McDonald et al. [36]calculated that reading the privacy policies you en-counter in a year would take 76 work days.Related Patterns: n/aPsychological Aspects: Even if the user is motivatedto read terms and conditions, missing opportunity tofully comprehend all details makes a System 1-based pro-cessing more probable.Strategies: Obscure

5.5 Immortal Accounts

Name/Aliases: Immortal AccountsContext: Many services require user accounts, eitherbecause they are necessary for service fulfilment, or be-cause user accounts represent a benefit for the service.Description: The service provider requires new usersto sign up for accounts to use the service. Once users de-cide to stop using the service, they might want to deletetheir accounts and associated data. However, the serviceprovider prevents the user from doing so by either—unnecessarily complicating the account deletion experi-ence, or by not providing any account deletion option

8 https://tosdr.org/9 https://web.archive.org/web/20001204110500/http://www.icq.com/legal/policy.html10 http://www.foxnews.com/tech/2010/04/15/online-shoppers-unknowingly-sold-souls.html

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 251

at all. Additionally, the service provider might trick theuser in the deletion process by pretending to delete theentire account, while still retaining (some) account data.Effect: When the user interface makes the accountdeletion options hard to access, the barrier to delete theaccount is increased. If the users are required to callthe customer support, the process is even more cum-bersome. Both of these deliberately inconvenient userexperiences may cause the user to reconsider the actualdeletion decision. A deletion process where the serviceprovider claims to remove the account, but instead justflags the user records as deleted while still keeping thedata gives the user a false feeling of deletion.Countermeasures: Online resources such as just-delete.me11 or accountkiller.com12 curate a list of ser-vice providers and their policies towards account re-moval. They provide step-by-step tutorials for users howto delete an account at those providers. If the service tobe used is known for a non-delete policy but requiresa user account, the usage of a throwaway account withincorrect data should be considered.Examples/Known Uses: As of February 2016, thecommunity-curated data set of justdelete.me lists 474services. 75 services thereof do not provide the possibil-ity to delete the account at all and 100 services requirecontacting the customer support. From the remaining299 services listed, another 31 services have a non-trivialdeletion process that requires additional steps.Related Patterns: The creation of accounts can berequired due to Forced Registration.Psychological Aspects: When the service providerrenders the user experience for account deletion delib-erately painful, users might struggle in the process. Ifthe user wants to delete the account, but fails to doso, cognitive dissonance may emerge. As a result, theuser could then reduce the inconsistent mental state byreconsidering their original intent and deciding not todelete the account.Strategies: Deny, Obscure

5.6 Address Book Leeching

Name/Aliases: Address Book LeechingContext A service provider offers users to upload orimport their address books to connect with known con-tacts on that service.

11 http://justdelete.me/12 http://www.accountkiller.com/

Description: When the user imports the list, the ser-vice executes a lookup against its own database. It thenprovides suggestions for connections to the user. How-ever, the service provider stores the list of all contacts asinternal data records for further processing—includingpurposes that have not been initially declared.Effect: Using an import feature may lead to expos-ing unwanted information, specifically the contents ofpersonal address books to third parties. A potential us-age of such information is the dispatch of invitations orother advertisements, at worst even in the name of theoriginal uploader without consent. Service provider maymisuse such data for profiling and tracking individualsthat do not yet possess a user account.Countermeasures: If it is unknown or unclear howa service provider is handling and processing importedcontact lists, such a feature should be avoided. Manymobile and desktop operating systems allow users todeny applications access to address book data. Usersshould routinely click on deny unless it is definitely re-quired or in their interest to share those data.Examples/Known Uses: In 2008, the social bookcataloging website goodreads.com attracted negative at-tention for unsolicited invite emails based on the addressbook import feature. The experiences of customers andreactions of the service providers are still available ona customer support page13. Based on a misleading up-load form design, users thought they would only providecontacts for matching against goodreads’ user base. In-stead, goodreads sent invite emails to persons which hadmail addresses not yet registered at goodreads, therebyreferring to the user who provided the address.Related Patterns: This pattern is a potential sourceof information for Shadow User Profiles.Psychological Aspects: Trading personal informationfor instant connections to friends or known contacts ismotivated by the need to belong.Strategies: Maximize, Preserve

5.7 Shadow User Profiles

Name/Aliases: Shadow User ProfilesContext: A service provider tracks personal informa-tion about individuals.Description: While registered users have deliberately

13 https://getsatisfaction.com/goodreads/topics/why_did_goodreads_trick_me_into_spamming_my_entire_address_book

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 252

opted in for a user account and an associated profile,the service provider may collect information and keeprecords about individuals that do not use the service.For instance, in a social network, the social graph canbe supplemented with persons that are not members ofthe network, but are known to the network based ondata from members (e.g., imported address books, con-tent metadata, or mentions). Such non-members enrichthe graph and improve the quality of algorithms suchas contact suggestions.Effect: The service provider stores and processes infor-mation on individuals without their knowledge or con-sent. The affected individuals are not aware of personaldata records they have accidentally created or that havebeen provided by third parties.Countermeasures: While it is possible to minimizethe own data trail, the accidental release of personaldata through third parties cannot always be prevented.Examples/Known Uses: The basic mechanism ofshadow user profiles fuels the entire online advertise-ment industry. Although not verifiable, social networksmay store informations of non-users. This notion isbased on the experiences of newly registered users ofsocial networks who received accurate friendship sug-gestions without having ever interacted with these per-sons on the social network before.Related Patterns: Address Book Leeching is a poten-tial source of information for this pattern.Psychological Aspects: Given the fact that this pat-tern operates without any knowledge of the affectedusers, it is not targeting any psychological aspects.Strategies: Maximize, Preserve, Centralize

6 ConclusionsIn this paper, we introduce the concepts of privacy darkstrategies and privacy dark patterns. Both are based onthe idea that actors intentionally manipulate people toprovide their personal data for collection, storage, andprocessing against their original intent and interest.

Documenting such strategies and patterns is a vitalfirst step towards a better recognition of such activities,e.g., in the Internet or in mobile apps. Our eight pri-vacy dark strategies Maximize, Publish, Centralize,Preserve, Obscure, Deny, Violate, and Fake pro-vide a coarse categorization for the subsequent patterns.Privacy dark patterns are documented using a uniformtemplate. Beyond a mere description of the pattern, the

template contains countermeasures and a psychologicalviewpoint that explains why the pattern is effective.

We extensively discussed psychological aspects inSection 4. Understanding those psychological mecha-nisms triggered by privacy dark patterns is of crucialimportance as it will allow affected users to take appro-priate countermeasures.

Based on our privacy dark pattern framework andthe extensive discussion of the related concepts, webriefly presented seven of such patterns including someconcrete examples. These patterns and more are avail-able in an extended form via an online privacy darkpattern portal dark.privacypatterns.eu. We have set upthis portal for the community to study and discuss ex-isting patterns and contribute new ones.

AcknowledgementsThe authors like to thank the anonymous reviewers fortheir valuable comments and suggestions to improve thequality of the paper. They are also grateful to YllkaThaqi and Florian Oberlies for insightful remarks andfruitful discussions.

References[1] G. Acar, C. Eubank, S. Englehardt, M. Juarez,

A. Narayanan, and C. Diaz, “The Web never forgets: Per-sistent tracking mechanisms in the wild,” in Proceedings ofthe 2014 ACM SIGSAC Conference on Computer and Com-munications Security. ACM, 2014, pp. 674–689.

[2] A. Acquisti, “Privacy in electronic commerce and the eco-nomics of immediate gratification,” in Proceedings of the5th ACM conference on Electronic commerce. ACM, 2004,pp. 21–29.

[3] ——, “Nudging privacy: The behavioral economics of per-sonal information.” IEEE Security & Privacy, vol. 7, no. 6,pp. 82–85, 2009.

[4] A. Acquisti, L. K. John, and G. Loewenstein, “The impactof relative standards on the propensity to disclose,” Journalof Marketing Research, vol. 49, no. 2, pp. 160–174, 2012.

[5] C. Alexander, S. Ishikawa, and M. Silverstein, A PatternLanguage: Towns, Buildings, Construction (Center for En-vironmental Structure Series). Oxford University Press,1977.

[6] H. Almuhimedi, F. Schaub, N. Sadeh, I. Adjerid, A. Ac-quisti, J. Gluck, L. F. Cranor, and Y. Agarwal, “Your loca-tion has been shared 5,398 times!: A field study on mobileapp privacy nudging,” in Proceedings of the 33rd AnnualACM Conference on Human Factors in Computing Systems,ser. CHI ’15. New York, NY, USA: ACM, 2015, pp. 787–796.

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 253

[7] Y. Amichai-Hamburger and E. Ben-Artzi, “Loneliness andinternet use,” Computers in Human Behavior, vol. 19, no. 1,pp. 71–80, 2003.

[8] C. M. Angst and R. Agarwal, “Adoption of electronic healthrecords in the presence of privacy concerns: The elaborationlikelihood model and individual persuasion,” MIS quarterly,vol. 33, no. 2, pp. 339–370, 2009.

[9] R. F. Baumeister and M. R. Leary, “The need to belong:desire for interpersonal attachments as a fundamental hu-man motivation.” Psychological Bulletin, vol. 117, no. 3, pp.497–529, 1995.

[10] K. Beck and W. Cunningham, “Using pattern languagesfor object oriented programs,” in Conference on Object-Oriented Programming, Systems, Languages, and Applica-tions (OOPSLA), 1987.

[11] H. Brignull, “Dark Patterns: fighting user deception world-wide,” http://darkpatterns.org/, accessed: 2016-01-24.

[12] A. Buchenscheit, B. Könings, A. Neubert, F. Schaub,M. Schneider, and F. Kargl, “Privacy implications of pres-ence sharing in mobile messaging applications,” in Proceed-ings of the 13th International Conference on Mobile andUbiquitous Multimedia. ACM, 2014, pp. 20–21.

[13] R. Cialdini, Influence : the psychology of persuasion. NewYork: Morrow, 1993.

[14] N. Doty and M. Gupta, “Privacy Design Patterns and Anti-Patterns,” in Trustbusters Workshop at the Symposium onUsable Privacy and Security, 2013.

[15] N. B. Ellison, C. Steinfield, and C. Lampe, “The benefitsof facebook "friends:" social capital and college students’use of online social network sites,” Journal of Computer-Mediated Communication, vol. 12, no. 4, pp. 1143–1168,2007.

[16] R. H. Fazio, “Multiple processes by which attitudes guidebehavior: The MODE model as an integrative framework,”Advances in Experimental Social Psychology, vol. 23, pp.75–109, 1990.

[17] L. Festinger, A theory of cognitive dissonance. Stanforduniversity press, 1962, vol. 2.

[18] M. Fowler, Patterns of Enterprise Application Architecture.Boston: Addison-Wesley Professional, 2003.

[19] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Designpatterns: elements of reusable object-oriented software.Pearson Education, 1994.

[20] H. Gangadharbatla, “Facebook me: Collective self-esteem,need to belong, and internet self-efficacy as predictors ofthe igeneration’s attitudes toward social networking sites,”Journal of interactive advertising, vol. 8, no. 2, pp. 5–15,2008.

[21] S. Gürses, C. Troncoso, and C. Diaz, “Engineering privacyby design,” Computers, Privacy & Data Protection, vol. 14,2011.

[22] M. Hafiz, “A collection of privacy design patterns,” in Pro-ceedings of the 2006 conference on Pattern languages ofprograms. ACM, 2006, p. 7.

[23] E. T. Higgins, Beyond pleasure and pain: How motivationworks. Oxford University Press, 2011.

[24] J.-H. Hoepman, “Privacy Design Strategies,” CoRR, vol.abs/1210.6621, 2012.

[25] G. Hohpe and B. Woolf, Enterprise Integration Patterns -Designing, Building, and Deploying Messaging Solutions,

1st ed. Boston: Addison-Wesley Professional, 2004.[26] D. J. Hughes, M. Rowe, M. Batey, and A. Lee, “A tale of

two sites: Twitter vs. Facebook and the personality predic-tors of social media usage,” Computers in Human Behavior,vol. 28, no. 2, pp. 561–569, 2012.

[27] P. Hustinx, “Privacy by design: delivering the promises,”Identity in the Information Society, vol. 3, no. 2, pp. 253–255, 2010.

[28] T. Jones, “Facebook’s "evil interfaces",” https://www.eff.org/de/deeplinks/2010/04/facebooks-evil-interfaces, ac-cessed: 2016-02-25.

[29] D. Kahneman, Thinking, fast and slow. Macmillan, 2011.[30] B. P. Knijnenburg and A. Kobsa, “Increasing sharing ten-

dency without reducing satisfaction: Finding the bestprivacy-settings user interface for social networks,” in Pro-ceedings of the International Conference on InformationSystems - Building a Better World through Information Sys-tems, ICIS 2014, Auckland, New Zealand, December 14-17,2014, 2014.

[31] B. P. Knijnenburg, A. Kobsa, and H. Jin, “Counteractingthe negative effect of form auto-completion on the privacycalculus,” in Thirty Fourth International Conference on In-formation Systems, Milan, 2013.

[32] A. Kobsa, H. Cho, and B. P. Knijnenburg, “The effect ofpersonalization provider characteristics on privacy attitudesand behaviors: An elaboration likelihood model approach,”Journal of the Association for Information Science and Tech-nology, 2016, in press.

[33] D. Laibson, “Golden eggs and hyperbolic discounting,” TheQuarterly Journal of Economics, vol. 112, no. 2, pp. 443–478, 1997.

[34] P. B. Lowry, G. Moody, A. Vance, M. Jensen, J. Jenkins,and T. Wells, “Using an elaboration likelihood approach tobetter understand the persuasiveness of website privacy as-surance cues for online consumers,” Journal of the AmericanSociety for Information Science and Technology, vol. 63,no. 4, pp. 755–776, 2012.

[35] E. Luger, S. Moran, and T. Rodden, “Consent for all: re-vealing the hidden complexity of terms and conditions,” inProceedings of the SIGCHI conference on Human factors incomputing systems. ACM, 2013, pp. 2687–2696.

[36] A. M. McDonald and L. F. Cranor, “Cost of reading privacypolicies, the,” ISJLP, vol. 4, p. 543, 2008.

[37] A. Nadkarni and S. G. Hofmann, “Why do people use Face-book?” Personality and Individual Differences, vol. 52, no. 3,pp. 243–249, 2012.

[38] N. Notario, A. Crespo, Y.-S. Martín, J. M. Del Alamo,D. Le Métayer, T. Antignac, A. Kung, I. Kroener, andD. Wright, “PRIPARE: Integrating Privacy Best Practicesinto a Privacy Engineering Methodology,” in Security andPrivacy Workshops (SPW), 2015 IEEE. IEEE, 2015, pp.151–158.

[39] R. E. Petty and J. T. Cacioppo, The elaboration likelihoodmodel of persuasion. Springer, 1986.

[40] S. Romanosky, A. Acquisti, J. Hong, L. F. Cranor, andB. Friedman, “Privacy patterns for online interactions,” inProceedings of the 2006 conference on Pattern languages ofprograms. ACM, 2006, p. 12.

[41] M. Schumacher, “Security patterns and security standards.”in EuroPLoP, 2002, pp. 289–300.

UnauthenticatedDownload Date | 10/23/16 6:36 PM

Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns 254

[42] T. Schümmer, “The public privacy–patterns for filteringpersonal information in collaborative systems,” in CHI2004:Proceedings of the Conference on Human Factors in Com-puting Systems, 2004.

[43] K. E. Stanovich and R. F. West, “Advancing the rationalitydebate,” Behavioral and Brain Sciences, vol. 23, no. 05, pp.701–717, 2000.

[44] F. Strack and R. Deutsch, “Reflective and impulsive deter-minants of social behavior,” Personality and Social Psychol-ogy Review, vol. 8, no. 3, pp. 220–247, 2004.

[45] R. Thaler, Nudge : improving decisions about health, wealth,and happiness. New York: Penguin Books, 2009.

[46] J. Tidwell, Designing Interfaces. Sebastopol: "O’ReillyMedia, Inc.", 2010.

[47] A. Tversky and D. Kahneman, “Judgment under uncer-tainty: Heuristics and biases,” science, vol. 185, no. 4157,pp. 1124–1131, 1974.

[48] J. van Rest, D. Boonstra, M. Everts, M. van Rijn, andR. van Paassen, Designing privacy-by-design. Springer,2014, pp. 55–72.

[49] K. D. Williams, C. K. Cheung, and W. Choi, “Cyberos-tracism: effects of being ignored over the internet.” Jour-nal of Personality and Social Psychology, vol. 79, no. 5, pp.748–762, 2000.

UnauthenticatedDownload Date | 10/23/16 6:36 PM