Date post: | 07-Jul-2016 |
Category: |
Documents |
Upload: | matthew-reach |
View: | 231 times |
Download: | 1 times |
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 1/47
Business Fax Solutions Send & Receive Faxes Online. HIPAA, SOX & GLB Compliant.
Tutorial: Step by Step Database Design in SQLFeb 22, 2015 23,195 views 67 Likes 7 Comments
Please check out my related article "How did the modern relational database
come to be?" which is currently trending in Big Data and follow me for daily
articles on technology, digital marketing, psychology and pharmaceuticals.
Database Design and Implementation is applicable for whatever industryyour in. Here is a step by step approach to designing and implementing adatabase in your organisation, using specific data from a sweet shopcase study I implemented during my M.Sc. in Software & InformationSystems. By the end of this tutorial, you will know about databases,advantages of databases system over regular file system, the steps of adatabase design process, software development lifecycle, qualities of awell built database, relations and relationships, data integrity, and more.Databases are used in every industry, including the pharmaceuticalindustry.Background of Databases
The database system approach to data management overcomes many of the
shortcomings of the oldfashioned file system approach. One of the key features
of a database system is that data is stored as a single logical unit. What this means
is that although the data may be spread across multiple physical files, the
David McCaldinMSc Student / Penetration Tester / Medical Devices / CSV Follow
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 2/47
database conveys the data as being located in a single data repository. Organizing
data in single logical repository allows for easy manipulation and querying of the
data, in contrast with traditional file systems where the programmer must specify
what and how the data retrieval is done.
With database systems, it need only be specified what must be done, the DBMS
(Database Management System) does the rest. Another advantage of the database
approach is that, because data is located in one single database, data in different
physical locations need not be duplicated. The database software can interact
with all the data in the database. Non duplication of data is one way of
maintaining the integrity of the data. When data is allowed to be duplicated,
errors can happen if one instance of the data is altered and another instance
remains the same. When data is allowed to be duplicated, more maintenance and
system resources are required to ensure that data is always integral.
One of the greatest benefits of databases is that data can be shared or secured
among users or applications. There is more control and accountability over how
the data is managed because the data all resides in one database.
If there are shortcomings to database systems, its that much more powerful and
sophisticated software is needed to control the database and designing the
software and database can be extremely time consuming. More extensive
knowledge of how to use the database is required, thus making the database
system less user friendly than traditional file systems. Since the database is one
logical repository, even a small error can damage the entire database and reduce
the integrity of the data. One benchmark of a good database is one which is
complete, integral, simple, understandable, flexible and implementable. Batini et
al says that database modelling strives for a “non redundant, unified
representation of all data managed in an organization”. By following the database
software development lifecycle methodology, and by using the data models, the
database design ideals are fulfilled and will minimize the disadvantages.
Databases & the Software Development Lifecycle
The steps in developing any application can be represented as a linear sequence
where each step in the sequence is a function, which passes its output to its
successor function. Adherence of a ‘waterfall development model’ ensures quality
software, which is ‘complete’, ‘efficient’, ‘usable’, ‘consistent’, ‘correct’ and
‘flexible’. These traits are also some of the core underpinnings of a wellbuilt
database. The waterfall model can be applied to database design theory as
effectively as it is applied to other software engineering theory. The steps can be
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 3/47
summarized as follows:
Requirements specification > Analysis > Conceptual design >Implementation Design> Physical Schema Design and Optimisation
In consultation with all potential users of the database, a database designer’s first
step is to draw up a data requirements document. The requirements document
contains a concise and nontechnical summary of what data items will be stored
in the database, and how the various data items relate to one another. Taking the
‘data requirements document’, further analysis is done to give meaning to the
data items, e.g. define the more detailed attributes of the data and define
constraints if needed. The result of this analysis is a ‘preliminary specifications’
document. Taking the specifications document, the database designer models
how the information is viewed by the database system and is how it is processed
and conveyed to the end user. In the implementation design phase, the
conceptual design is translated into a more lowlevel, DBMS specific design.
Data Models & Schemas as a Means of Capturing Data
The database development design phases brings up the concept of ‘data models’.
Data models are diagrams or schemas, which are used to present the data
requirements at different levels of abstraction. The first step in the Database
Development Life Cycle is to draw up a requirements document.
Figure 1: A basic example of a requirements document
The requirements document can then be analysed and turned into a basic data set
(as shown in Figure 2) which can be converted into a conceptual model. The end
result of the conceptual design phase is a conceptual data model (Figure 3), which
provides little information of how the database system will eventually be
implemented. The conceptual data model is simply a highlevel overview of the
database system.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 4/47
Figure 2: A Database Data Set is the Result of analyzing the Information from
the Requirements Phase. The Primary Keys are Underlined.
Figure 3: A Normalized EntityRelationship model (ERD) in Crow’s Foot
Notation is an Example of a Conceptual Data Model and provides no
information of how the database system will eventually be implemented
In the implementation design phase, the conceptual data model is translated into
a ‘logical’ representation of the database system. The logical data model conveys
the “logical functioning and structure” of the database and describes ‘how the
data is stored’ (e.g. what tables are used, what constraints are applied) but is not
specific to any DBMS. Logical database model is a lowerlevel conceptual model,
which must be translated to a physical design.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 5/47
Figure 4: In the implementation design phase, the conceptual data model (ERD)
is translated into a ‘logical’ representation (logical schema) of the database
system: a data dictionary.
Physical modelling deals with the “representational aspects” and the “operational
aspects” of the database, i.e. the internal DBMS specific operations and processes
and how the DBMS interacts with the data, the database and the user. The
translation from logical design to physical design assigns “functions” to both the
machine (the DBMS) and to the user, functions such as storage and security, and
additional aspects such as consistency (of data) and learnability are dealt with in
the physical model/schema. Practically speaking, a physical schema is the SQL
code used to build the database.
One benchmark of a good database is one, which is complete, integral, simple,
understandable, flexible and implementable. Database modelling strives for a
nonredundant, unified representation of all data managed in an organization. By
following the above methodology, and by using the data models, these database
design ideals are fulfilled. In conclusion, here are two examples of why using data
models is paramount to capturing and conveying data requirements of the
information system:
1. By drawing up a ‘logical model’, extra data items can be added more easily in
that model then in the physical model. A database design that can change
easily according to needs of the company is important, because it ensures the
final database system is complete and uptodate.
2. Another consideration is understandability. By initially creating a ‘conceptual
model’, both the designer and the organization are able to understand the
database design and decide if it is complete or not. If there were no
‘conceptual model’, the organization would not be able to ‘conceptualize’ the
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 6/47
database design and make sure that it actually represents all the data
requirements of the organization.
3. By creating a physical model, the designers can have a low level overview of
how the database system would operate before it is actually implemented.
SQL Statements – Implementing the Database
The final step is to physically implement the logical design which was illustrated
in Figure 4. To physically implement the database, SQL can be used. These are
the main steps in implementing the database:
1. Create the Database Tables
The tables come directly from the information contained in the Data Dictionary.
The following blocks of code each represent a row in the data dictionary and are
executed one after another. The blocks of “create table” code contain the details of
all the data items (COMPANY, SUPPLIER, PURCHASES, EMPLOYEE etc), their
attributes (names, ages, costs, numbers and other details), the Relationships
between the data items, the Keys and Data Integrity Rules. All of this information
is already detailed in the Data Dictionary, but now we are actually converting it
and implementing it in a physical database system.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 7/47
Explanation:
1. The create table statement indicates that you want a table to be created.
2. The name of the table proceeds the first '('
3. The table attributes and Data Integrity Rules are defined within the two
parentheses. How the table relates to other tables is also defined within the
two parentheses (e.g. by defining FOREIGN KEYS)
4. The not null statement means that if you try to populate the table with
values, but leave the value of that attribute empty, you will get an error.
5. The varchar2 (19) means a string of 19 characters.
6. The number (6,2) means a number which can have a number of up to 6
digits, 2 of them being after the decimal place, e.g. you can have a number
from 0.0 to 1234.56.
7. The date means that that attribute will be represented as a date within the
Tutorial: Step by Step DatabaseDesign in SQLDavid McCaldin
Google dodges a $9 billionbullet; The Ashley Madisonfiles: adulterers are bad boys …business, too; and more newsJohn C Abell
Facebook and others thinkmessaging as a platform is thenext big thing. Are they right?Chris Moore
Pulse
What is LinkedIn? Join Today Sign In
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 8/47
database system.
8. The CONSTRAINT statement means that a constraint is being defined. This
statement will be used to describe which of the attributes are primary keys
and which (if any) of the attributes are foreign keys (referencing another
table).
9. The CONSTRAINT statement is in the form CONSTRAINT xxx PRIMARY
KEY(name_of_attribute_that_you_want_as_the_primary_key) or
CONSTRAINT yyy FOREIGN
KEY(name_of_attribute_that_references_another_table) REFERENCES
hhh(name_of_attribute_that_references_another_table) where the values
of xxx and yyy are just arbitrarily made up names which are not important.
hhh is the name of the base table being referenced.
2. Populate the tables
Use SQL statements to populate each table with specific data (such as employee
names, ages, wages etc).
3. Query the database.
Write SQL statements to obtain information and knowledge about the company,
e.g. how many employees are there, total profit etc.
Keys & Data Integrity Rules
Data integrity rules are a core component of a data model. Integrity rules
“implicitly or explicitly define the set of consistent database state(s)”. So, integrity
rules “ensure that database states and changes of state confirm to specified rules.”
Data integrity rules are of two types: Entity integrity rules and Referential
integrity rules..
How do keys relate to ensuring that changes in database states confirm to
specified rules?
Well, for example, you could ensure that the primary key of an entity cannot be
null. This is one way of ensuring entity integrity. If primary keys were allowed to
be null, then there would be no way of ensuring that individual entities were
uniquely identifiable. If you cannot ensure that individual entities are uniquely
identifiable then you can’t ensure that the database is ‘integral’, which is a core
property of a properly designed data model. So, by ensuring that keys follow
certain rules, you can ensure integrity of data.
How Technology HijacksPeople’s Minds — from aMagician and Google’s Desig…EthicistTristan Harris
CEO pay isn't slowing;Investors can't get enough ofnow$18 billion Snapchat, an…more newsIsabelle Roughol
Hiring Managers: Stop Insultingthe Job CandidatesBrian de Haaff
Here’s Why You’re Wrong ToThrow Shade At The TSA AndTheir Fired Executive
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 9/47
Written byDavid McCaldin
7 comments
Another way of enforcing integrity of data via keys, is to ensure that, if two tables
are related to each other, an attribute of one relation must be the same as the
primary attribute (primary key) of the other one. Enforcing this rule ensures
referential integrity of data.
So, we do need integrity rules, and proper defining of keys are a means of
enforcing them.
Relationships
When initially explaining the ‘relational model’, E.F. Codd proposed that users
should be abstracted from the internal representation of the data, such that if the
internal representation of that data were to change (e.g. because of system
growth), the way that the user perceives the data should remain unchanged. This
is why he proposed that users should only interact with a “collection of time
varying relationships” i.e. a user should only know the “name of the relationship
together with its domain” (e.g. department is the domain of employees, and the
employees are ‘owned’ by the department) rather than the relation (table) itself.
In using the terms ‘relation’ and ‘table’ as synonyms, Codd must have implied
that a table should be viewed in terms of its ‘relationship’ with other tables.
Relationships are what bind the relations/tables in a database together, so proper
understanding is needed.
Incorrect understanding of relationships may lead to incorrectly defined
relationships between tables. Incorrectly defined relationships between tables
could lead to data not being updated correctly in some tables, or could cause a
data item to be unnecessarily duplicated in another table. All this may lead to
incomplete ‘information’ in the database, which in turn results in incomplete
‘knowledge’.
Featured In Big Data
Databases, Software Development, Data Analysis
Follow
Like Comment 67 likes
Add your comment
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 10/47
Google dodges a $9 billion bullet; The AshleyMadison files: adulterers are bad boys in business,too; and more newsMay 26, 2016 59,720 views 494 Likes 17 Comments
Facebook and Microsoft are building what will be "the highest
capacity subsea cable to ever cross the Atlantic." There are hundreds, for
Huda Naiem Mohamed OsmanInterpreter: Language Resources, Tax Returns' Preparer,Programmer/Developer of Database/System
David McCaldin :) I am speechless towards your conise course onDatabases. You have managed to narrate the knowledge and conceptsfluently and captively. You have excelled in your choice of models todemonstrate how can one make an efficient database file. Output display isessential and will follow naturally, once the foundation is set correctly. Allrelevant program codes and queries will need learning, but all these willlead you nowhere if you do not know how to initiate your data system andwarehouse TO WORK ON with the latter coding, queries and reporting.
Thank you on behalf of my peer readers and on behalf of myself for thetreasured input.
Like(3) Reply February 24, 2015
Hamza Tariq, David McCaldin, and OLAYEMI OYEWALE
Show More
John C AbellManaging Editor, News at LinkedIn Ex Reuters Ex Wired Follow
Popular
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 11/47
the record, but demand keeps growing. This one should be done by October 2017,
CNBC reports. Data infrastructure buildout is now the purview of suppliers, not
telecommunication intermediaries; two years ago Alphabet announced a Pacific
cable project with five Asiabased telecom companies.
So Facebook and Microsoft are going to build a giant underseacable across the Atlantic—oh, and it's 160TB/sec bit.ly/249ApCV1:08 PM 26 May 2016
262 176
WIRED @WIRED
Facebook and Microsoft Are Laying a Giant Cable Across th…Internet giants are starting to build enormous networks of their own,taking over the role traditionally played by telecom companies.wired.com
Follow
More evidence the 2017 iPhone will have a (mostly) glass case. The
chairman of "long time iPhone chassis maker Catcher Technology" told his
annual meeting it's a sure thing, reports Forbes. It would be reinforced with a
steel case, which we think could double (uh oh) as an antenna.
Google prevailed in a $9 billion lawsuit brought Oracle alleging the
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 12/47
Written byJohn C Abell
17 comments
search giant had violated its Java copyright to develop Android. This
case has bounced around various courts for years (the Supreme Court declined to
hear the federal one), and Oracle said it would appeal the verdict, by a California
state jury. For the nonelementary version, read Greg Leffler's analysis.
Make of this what you will: A relatively small number of Ashley Madison
members registered with work addresses. But it was enough for researchers to
draw some conclusions about their companies. On the downside: They were were
likely to have been disciplined by the SEC, and were above average in "bribery
and fraud scandals, tax disputes, human rights violations and product quality
problems," observes Financial Times columnist Michael Skapinker. On the
upside: These establishments "also appeared to be more inventive and creative"
— above average in patents, which "covered a wider range of technologies," and
"tended to be in highgrowth sectors."
Donald Trump clinched the Republican presidential nomination with
1,239 delegates, according to a count first reported by the AP.
Cover Art: Illustrations of herostyle G7 leaders built for a NGO to encourage
them to be Universal Health Coverage superheroes, Ise city, Mie prefecture on
May 26, 2016. World leaders kick off two days of G7 talks in Japan on May 26
with the creaky global economy, terrorism, refugees, China's controversial
maritime claims, and a possible Brexit headlining their packed agenda.
Featured In Technology, Daily Digest
Follow
Like Comment 497 likes
Andy BouraSenior Information Security Architect Thomson Reuters (Director SecureProduct Architecture | CISSP | MPhys )
Justice in Google vs Oracle. I read the judges notes from the previous trial itwas pretty clear from that Oracle were having a punt to see if they could getanything. It's really important precident for software engineering based onmy understanding APIs are not copyrightable.
Add your comment
Popular
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 13/47
Facebook and others think messaging as a platformis the next big thing. Are they right?May 26, 2016 10,461 views 236 Likes 26 Comments
Like(5) Reply(4) 17 hours ago
Suzy Ekman, Diego Valero Seyffert, Beñat Galdós Aizpurua, +2
Andy BouraSenior Information Security Architect Thomson Reuters(Director Secure Product Architecture | CISSP | MPhys )
To be clear. Google didn't develope an application in Java that would have been fine. They created a whole new virtualmachine to run Android Java applications without payingOracle for their virtual machine.
Like 5 hours ago
Khaled HussainSenior Systems Engineer at GlobalCharge
Erik Grob I think they'd absolutely want you to develop yoursoftware in Java on the premise that if your software turns outto be successful, they'd take "a punt to see if they could getanything" from you, as Andy Boura speculates. Perhaps nowthey won't...
Like 5 hours ago
Show More
Show More
Chris MooreFollow
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 14/47
As investors in the mobile ecosystem, we are constantly working to better
understand and predict how the unique aspects of the mobile computing platform
will allow entrepreneurs to build new and lasting businesses across industries.
Inspired by recent industry developments and conversations we’ve been having
with friends and founders creating mobilefirst services, we decided to open
source more of our conversations and create a forum to surface different
perspectives on mobile related issues that have relevance across a range of
categories.
Our intent is to share opinions and practical advice around what’s working as well
as what’s not. We’d love for more folks to join the discussion as our hope is that
this process will accelerate our collective knowledge. Look for a mix of industry
commentary, metrics and analysis, guest posts and quick takes in this vein (like
this perspective on Gboard from my partner Jamie Davidson).
Lately we’ve been awash in the messagingasaplatform discussions as all the
major players are coming out with messaging + AI plays. Last week Google
announced Allo as a kind of new smart messaging app. Facebook’s F8 messaging
news had already been top of mind as both a source of promise and a painful
reminder of how challenging the current app discovery and distribution structure
really is. Could messaging bots become a new distribution channel for app
providers? The people we’ve surveyed are intrigued but skeptical. Intrigued for
the obvious reason that everyone is looking for another way to reach consumers
where they’re spending time. But skeptical based on the applicability of the chat
based interface for the full range of apps, and the inevitability of the same
discovery challenges if the model gains awareness and traction.
One app whose approach to distribution and engagement could inform us more
broadly is Riffsy (also a Redpoint portfolio company). At the recent Mobile Apps
Unlocked conference (an awesome event organized by Grow.co founders Adam
Lovallo and Jay Weintraub) I did an onstage interview with Rffsy founder and
CEO David McIntosh where we talked about Riffsy’s approach to leveraging
messaging as a platform. David told the congregation of mobile app thinkers that
making Riffsy available across multiple messaging platforms was key to achieving
scale and encouraging repeat users. Riffsy works well on iOS, Kik, Twitter, and
Facebook Messenger among others. Its user base has shared tens of billions of
GIFs across multiple mobile messaging platforms no small feat given the
challenges of platform fragmentation. App makers here in the U.S. can’t borrow
from the Tencent playbook where the WeChat platform dominates the messaging
app landscape and serves as the de facto portal for 3rd party applications. But I’d
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 15/47
Written byChris Moore
26 comments
argue that makes the whole landscape more interesting – and an area we think is
still in its early stages here in the U.S.
Will messenger as a platform be a THING? And what does that portend for all the
platforms and app makers competing for our attention as well as the mobile
ecosystem at large? As we’re actively exploring other ideas to tackle the problem
of getting and keeping people’s attention in this crowded mobile landscape, we’re
taking a fresh look at the app as we know it. The time is right for new ideas about
messaging as a platform and how we can build an effective distribution and
engagement model for mobile content and services.
Featured In Technology, Entrepreneurship, Editor's Picks, VC & Private Equity, Mobile
Venture Capital, Mobile Applications, StartUps
Follow
Like Comment 236 likes
Mark ParrishSr. IT Project Manager, Product Development at TMG Health
Yawn! A fine example of a solution in search of a problem. Another intrusioninto being a true human that interacts with society as needed. As MontyPython says, this will increase " SPAM SPAM SPAM". Do businesses reallyneed to be attached to customers via messaging apps?
Like(3) Reply(3) 19 hours ago
Jonathan de Potter, Brian Hochstetter, and Robert W. Kapfhammer
Taitusi VuatakiProject Coordinator
Nope just Facebook
Like 5 hours ago
Sr. IT Project Manager, Product Development at TMG Health
It's very much missing the point as a solution... the problem,that is alluded to is that they want a mechanism that will getpeople to buy things via a chat system; the bottleneck with thatis the psychology of buying. Until this aspect is fullycharacterised, then Dallas is completely correct. For me thesolution is not the platform, but the mechanism of purchase. Aplatform for sales (or maybe "arena") could be a supermarket,but the mechanism is the till; influencing factors are the
Add your comment
Popular
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 16/47
How Technology Hijacks People’s Minds — from aMagician and Google’s Design EthicistMay 26, 2016 7,013 views 354 Likes 39 Comments
I’m an expert on how technology hijacks our psychological vulnerabilities. That’s
why I spent the last three years as a Design Ethicist at Google caring about how to
design things in a way that defends a billion people’s minds from getting hijacked.
When using technology, we often focus optimistically on all the things it does for
us. But I want you to show you where it might do the opposite.
ambience (music, temperature, aroma, colours), the absenceof clocks, the layout of the products, etc... you could argue thata solution mechanism might be having the till fitted to thetrolley you push around, or at the end of each aisle, etc... Thereal platform, might be the trolley or the manned till and selfservice till, and is very much a dependency of the mechanism.That's my tuppence. All this wideeyed IoT overethusiasm issuffering from a failure of problem analysis and requirementscapture.
Like(2) 7 hours ago
Kristian Wilde and Dallas Stephens
Show More
Show More
Tristan HarrisDesign Ethics & Product Philosopher at Google Follow
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 17/47
Where does technology exploit our minds’ weaknesses?
I learned to think this way when I was a magician. Magicians start by looking for
blind spots, edges, vulnerabilities and limits of people’s perception, so they can
influence what people do without them even realizing it. Once you know how to
push people’s buttons, you can play them like a piano.
(That’s me performing sleight of hand magic at my mother’s birthday party)
And this is exactly what product designers do to your mind. They play your
psychological vulnerabilities (consciously and unconsciously) against you in the
race to grab your attention.
I want to show you how they do it.
Hijack #1: If You Control the Menu, You Control the Choices
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 18/47
Western Culture is built around ideals of individual choice and freedom. Millions
of us fiercely defend our right to make “free” choices, while we ignore how those
choices are manipulated upstream by menus we didn’t choose in the first place.
This is exactly what magicians do. They give people the illusion of free choice
while architecting the menu so that they win, no matter what you choose. I can’t
emphasize enough how deep this insight is.
When people are given a menu of choices, they rarely ask:
“what’s not on the menu?”
“why am I being given these options and not others?”
“do I know the menu provider’s goals?”
“is this menu empowering for my original need, or are the choices actually a
distraction?” (e.g. an overwhelmingly array of toothpastes)
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 19/47
(How empowering is this menu of choices for the need, “I ran out of
toothpaste”?)
For example, imagine you’re out with friends on a Tuesday night and want to
keep the conversation going. You open Yelp to find nearby recommendations and
see a list of bars. The group turns into a huddle of faces staring down at their
phones comparing bars. They scrutinize the photos of each, comparing cocktail
drinks. Is this menu still relevant to the original desire of the group?
It’s not that bars aren’t a good choice, it’s that Yelp substituted the group’s
original question (“where can we go to keep talking?”) with a different question
(“what’s a bar with good photos of cocktails?”) all by shaping the menu.
Moreover, the group falls for the illusion that Yelp’s menu represents acomplete
set of choices for where to go. While looking down at their phones, they don’t see
the park across the street with a band playing live music. They miss the popup
gallery on the other side of the street serving crepes and coffee. Neither of those
show up on Yelp’s menu.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 20/47
(Yelp subtly reframes the group’s need “where can we go to keep talking?” in
terms of photos of cocktails served.)
The more choices technology gives us in nearly every domain of our lives
(information, events, places to go, friends, dating, jobs) — the more we assume
that our phone is always the most empowering and useful menu to pick from. Is
it?
The “most empowering” menu is different than the menu that has the
most choices. But when we blindly surrender to the menus we’re given, it’s easy
to lose track of the difference:
“Who’s free tonight to hang out?” becomes a menu of most recent people
who texted us (who we could ping).
“What’s happening in the world?” becomes a menu of news feed stories.
“Who’s single to go on a date?” becomes a menu of faces to swipe on Tinder
(instead of local events with friends, or urban adventures nearby).
“I have to respond to this email.” becomes a menu of keys to type a response
(instead of empowering ways to communicate with a person).
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 21/47
When we wake up in the morning and turn our phone over to see a list of
notifications — it frames the experience of “waking up in the morning” around a
menu of “all the things I’ve missed since yesterday.” (for more examples, see Joe
Edelman’s Empowering Design talk)
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 22/47
By shaping the menus we pick from, technology hijacks the way we perceive our
choices and replaces them with new ones. But the closer we pay attention to the
options we’re given, the more we’ll notice when they don’t actually align with our
true needs.
Hijack #2: Put a Slot Machine In a Billion Pockets
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 23/47
If you’re an app, how do you keep people hooked? Turn yourself into a slot
machine.
The average person checks their phone 150 times a day. Why do we do this? Are
we making 150 conscious choices?
One major reason why is the #1 psychological ingredient in slot
machines:intermittent variable rewards.
If you want to maximize addictiveness, all tech designers need to do is link a
user’s action (like pulling a lever) with a variable reward. You pull a lever and
immediately receive either an enticing reward (a match, a prize!) or nothing.
Addictiveness is maximized when the rate of reward is most variable.
Does this effect really work on people? Yes. Slot machines make more
money in the United States than baseball, movies, and theme
parkscombined. Relative to other kinds of gambling, people get
‘problematically involved’ with slot machines 3–4x faster according to NYU
professor Natasha Dow Schull, author of Addiction by Design.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 24/47
But here’s the unfortunate truth — several billion people have a slot
machine their pocket:
When we pull our phone out of our pocket, we’re playing a slot machineto
see what notifications we got.
When we pull to refresh our email, we’re playing a slot machine to see what
new email we got.
When we swipe down our finger to scroll the Instagram feed, we’replaying a
slot machine to see what photo comes next.
When we swipe faces left/right on dating apps like Tinder, we’re playing a
slot machine to see if we got a match.
When we tap the # of red notifications, we’re playing a slot machine to
what’s underneath.
Apps and websites sprinkle intermittent variable rewards all over their products
because it’s good for business.
But in other cases, slot machines emerge by accident. For example, there is no
malicious corporation behind all of email who consciously chose to make it a slot
machine. No one profits when millions check their email and nothing’s there.
Neither did Apple and Google’s designers want phones to work like slot
machines. It emerged by accident.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 25/47
But now companies like Apple and Google have a responsibility to reduce these
effects by converting intermittent variable rewards into less addictive, more
predictable ones with better design. For example, they could empower people to
set predictable times during the day or week for when they want to check “slot
machine” apps, and correspondingly adjust when new messages are delivered to
align with those times.
Hijack #3: Fear of Missing Something Important (FOMSI)
Another way apps and websites hijack people’s minds is by inducing a “1% chance
you could be missing something important.”
If I convince you that I’m a channel for important information, messages,
friendships, or potential sexual opportunities — it will be hard for you to turn me
off, unsubscribe, or remove your account — because (aha, I win) you might miss
something important:
This keeps us subscribed to newsletters even after they haven’t delivered
recent benefits (“what if I miss a future announcement?”)
This keeps us “friended” to people with whom we haven’t spoke in ages
(“what if I miss something important from them?”)
This keeps us swiping faces on dating apps, even when we haven’t even met
up with anyone in a while (“what if I miss that one hot match who likes me?”)
This keeps us using social media (“what if I miss that important news story or
fall behind what my friends are talking about?”)
But if we zoom into that fear, we’ll discover that it’s unbounded: we’ll always
miss something important at any point when we stop using something.
There are magic moments on Facebook we’ll miss by not using it for the 6th
hour (e.g. an old friend who’s visiting town right now).
There are magic moments we’ll miss on Tinder (e.g. our dream romantic
partner) by not swiping our 700th match.
There are emergency phone calls we’ll miss if we’re not connected 24/7.
But living moment to moment with the fear of missing something isn’t how we’re
built to live.
And it’s amazing how quickly, once we let go of that fear, we wake up from the
illusion. When we unplug for more than a day, unsubscribe from those
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 26/47
notifications, or go to Camp Grounded — the concerns we thought we’d have don’t
actually happen.
We don’t miss what we don’t see.
The thought, “what if I miss something important?” is generated in advance of
unplugging, unsubscribing, or turning off — not after. Imagine if tech companies
recognized that, and helped us proactively tune our relationships with friends and
businesses in terms of what we define as “time well spent” for our lives, instead of
in terms of what we might miss.
Hijack #4: Social Approval
We’re all vulnerable to social approval. The need to belong, to be approved or
appreciated by our peers is among the highest human motivations. But now our
social approval is in the hands of tech companies.
When I get tagged by my friend Marc, I imagine him making a conscious choice to
tag me. But I don’t see how a company like Facebook orchestrated his doing that
in the first place.
Facebook, Instagram or SnapChat can manipulate how often people get tagged in
photos by automatically suggesting all the faces people should tag (e.g. by
showing a box with a 1click confirmation, “Tag Tristan in this photo?”).
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 27/47
So when Marc tags me, he’s actually responding to Facebook’s suggestion, not
making an independent choice. But through design choices like this,Facebook
controls the multiplier for how often millions of people experience their social
approval on the line.
(Facebook uses automatic suggestions like this to get people to tag more people,
creating more social externalities and interruptions.)
The same happens when we change our main profile photo — Facebook knows
that’s a moment when we’re vulnerable to social approval: “what do my friends
think of my new pic?” Facebook can rank this higher in the news feed, so it sticks
around for longer and more friends will like or comment on it. Each time they like
or comment on it, we’ll get pulled right back.
Everyone innately responds to social approval, but some demographics
(teenagers) are more vulnerable to it than others. That’s why it’s so important to
recognize how powerful designers are when they exploit this vulnerability.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 28/47
Hijack #5: Social Reciprocity (Titfortat)
You do me a favor — I owe you one next time.
You say, “thank you”— I have to say “you’re welcome.”
You send me an email— it’s rude not to get back to you.
You follow me — it’s rude not to follow you back. (especially for teenagers)
We are vulnerable to needing to reciprocate others’ gestures. But as with Social
Approval, tech companies now manipulate how often we experience it.
In some cases, it’s by accident. Email, texting and messaging apps are social
reciprocity factories. But in other cases, companies exploit this vulnerability on
purpose.
LinkedIn is the most obvious offender. LinkedIn wants as many people creating
social obligations for each other as possible, because each time they reciprocate
(by accepting a connection, responding to a message, or endorsing someone back
for a skill) they have to come back to linkedin.com where they can get people to
spend more time.
Like Facebook, LinkedIn exploits an asymmetry in perception. When you receive
an invitation from someone to connect, you imagine that person making a
conscious choice to invite you, when in reality, they likely unconsciously
responded to LinkedIn’s list of suggested contacts. In other words, LinkedIn
turns your unconscious impulses (to “add” a person) into new social obligations
that millions of people feel obligated to repay. All while they profit from the time
people spend doing it.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 29/47
Imagine millions of people getting interrupted like this throughout their day,
running around like chickens with their heads cut off, reciprocating each other —
all designed by companies who profit from it.
Welcome to social media.
After accepting an endorsement, LinkedIn takes advantage of your bias to
reciprocate by offering *four* additional people for you to endorse in return.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 30/47
Imagine if technology companies had a responsibility to minimize social
reciprocity. Or if there was an independent organization that represented the
public’s interests — an industry consortium or an FDA for tech — that monitored
when technology companies abused these biases?
Hijack #6: Bottomless bowls, Infinite Feeds, and Autoplay
Another way to hijack people is to keep them consuming things, even when they
aren’t hungry anymore.
How? Easy. Take an experience that was bounded and finite, and turn it into a
bottomless flow that keeps going.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 31/47
Cornell professor Brian Wansink demonstrated this in his study showing you can
trick people into keep eating soup by giving them a bottomless bowl that
automatically refills as they eat. With bottomless bowls, people eat 73% more
calories than those with normal bowls and underestimate how many calories they
ate by 140 calories.
Tech companies exploit the same principle. News feeds are purposely designed to
autorefill with reasons to keep you scrolling, and purposely eliminate any reason
for you to pause, reconsider or leave.
It’s also why video and social media sites like Netflix, YouTube or
Facebookautoplay the next video after a countdown instead of waiting for you to
make a conscious choice (in case you won’t). A huge portion of traffic on these
websites is driven by autoplaying the next thing.
Tech companies often claim that “we’re just making it easier for users to see the
video they want to watch” when they are actually serving their business interests.
And you can’t blame them, because increasing “time spent” is the currency they
compete for.
Instead, imagine if technology companies empowered you to consciously bound
your experience to align with what would be “time well spent” for you. Not just
bounding the quantity of time you spend, but the qualities of what would be
“time well spent.”
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 32/47
Hijack #7: Instant Interruption vs. “Respectful” Delivery
Companies know that messages that interrupt people immediately are more
persuasive at getting people to respond than messages delivered asynchronously
(like email or any deferred inbox).
Given the choice, Facebook Messenger (or WhatsApp, WeChat or SnapChat for
that matter) would prefer to design their messaging system to interrupt
recipients immediately (and show a chat box) instead of helping users respect
each other’s attention.
In other words, interruption is good for business.
It’s also in their interest to heighten the feeling of urgency and social reciprocity.
For example, Facebook automatically tells the sender when you “saw” their
message, instead of letting you avoid disclosing whether you read it(“now that
you know I’ve seen the message, I feel even more obligated to respond.”)
By contrast, Apple more respectfully lets users toggle “Read Receipts” on or off.
The problem is, maximizing interruptions in the name of business creates a
tragedy of the commons, ruining global attention spans and causing billions of
unnecessary interruptions each day. This is a huge problem we need to fix with
shared design standards (potentially, as part of Time Well Spent).
Hijack #8: Bundling Your Reasons with Their Reasons
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 33/47
Another way apps hijack you is by taking your reasons for visiting the app (to
perform a task) and make them inseparable from the app’s business
reasons(maximizing how much we consume once we’re there).
For example, in the physical world of grocery stories, the #1 and #2 most popular
reasons to visit are pharmacy refills and buying milk. But grocery stores want to
maximize how much people buy, so they put the pharmacy and the milk at the
back of the store.
In other words, they make the thing customers want (milk, pharmacy)
inseparable from what the business wants. If stores were truly organized to
support people, they would put the most popular items in the front.
Tech companies design their websites the same way. For example, when you you
want to look up a Facebook event happening tonight (your reason) the Facebook
app doesn’t allow you to access it without first landing on the news feed (their
reasons), and that’s on purpose. Facebook wants to convert every reason you
have for using Facebook, into their reason which is to maximize the time you
spend consuming things.
In an ideal world, apps would always give you a direct way to get what you want
separately from what they want.
Imagine a digital “bill of rights” outlining design standards that forced the
products used by billions of people to support empowering ways for them to
navigate toward their goals.
Hijack #9: Inconvenient Choices
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 34/47
We’re told that it’s enough for businesses to “make choices available.”
“If you don’t like it you can always use a different product.”
“If you don’t like it, you can always unsubscribe.”
“If you’re addicted to our app, you can always uninstall it from your phone.”
Businesses naturally want to make the choices they want you to make easier,
and the choices they don’t want you to make harder. Magicians do the same
thing. You make it easier for a spectator to pick the thing you want them to pick,
and harder to pick the thing you don’t.
For example, NYTimes.com lets you “make a free choice” to cancel your digital
subscription. But instead of just doing it when you hit “Cancel Subscription,” they
send you an email with information on how to cancel your account by calling a
phone number that’s only open at certain times.
Instead of viewing the world in terms of availability of choices, we should view
the world in terms of friction required to enact choices. Imagine a world where
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 35/47
choices were labeled with how difficult they were to fulfill (like coefficients of
friction) and there was an independent entity — an industry consortium or non
profit — that labeled these difficulties and set standards for how easy navigation
should be.
Hijack #10: Forecasting Errors, “Foot in the Door” strategies
Summary And How We Can Fix This
Are you upset that technology hijacks your agency? I am too. I’ve listed a few
techniques but there are literally thousands. Imagine whole bookshelves,
seminars, workshops and trainings that teach aspiring tech entrepreneurs
techniques like these. Imagine hundreds of engineers whose job every day is to
invent new ways to keep you hooked.
The ultimate freedom is a free mind, and we need technology that’s on our team
to help us live, feel, think and act freely.
We need our smartphones, notifications screens and web browsers to be
exoskeletons for our minds and interpersonal relationships that put our values,
not our impulses, first. People’s time is valuable. And we should protect it with
the same rigor as privacy and other digital rights.
Tristan Harris was a Product Philosopher at Google until 2016 where he studied
how technology affects a billion people’s attention, wellbeing and behavior. For
more resources on Time Well Spent, see http://timewellspent.io.
UPDATE: The first version of this post lacked acknowledgements to
those who inspired my thinking over many years including Joe
Edelman, Aza Raskin, Raph D’Amico, Jonathan Harris and Damon
Horowitz.
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 36/47
Written byTristan Harris
My thinking on menus and choicemaking are deeply rooted in Joe
Edelman’s work on Human Values and Choicemaking.
Featured In Entrepreneurship, Design, Social Media, Editor's Picks
Follow
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 37/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 38/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 39/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 40/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 41/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 42/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 43/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 44/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 45/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 46/47
5/27/2016 Tutorial: Step by Step Database Design in SQL
https://www.linkedin.com/pulse/tutorialstepdatabasedesignsqldavidmccaldin 47/47