N=billions: The smartphone revolution in the behavioral sciences
Geoffrey Miller
Almost everyone in the world will soon be able to participate in any behavioral research – remotely, electronically, continually
2012: • 7.0 billion people total• >5.0 billion mobile phone
users• 1.1 billion in China• 900 million in India• 700 million in Africa
Smartphone users:• 2012: 1.1 billion• 2015: 3 billion• 2020: 5 billion (ITU, 2012; Erickson, 2012)
Smartphone R&D: A windfall for behavioral sciences
Revenue 2012: – $1.5 trillion telecomms services– $200 billion smartphone sales– cf. Pharmaceuticals $970 billion
Smartphone R&D spending 2012– Samsung $6 billion– Nokia $1.7 billion– Apple $1 billion
cf. behavioral science research:– NIMH $1.5 billion– NSF $250 million
Ideal method for gathering data if psychology was invented today?
Smartphones.• general-purpose computer + sensor
array + GPS system + media capture system (+ small enough to hold to your head for phone calls)
• Ubiquitous, unobtrusive, intimate• Remotely accessible through
participants downloading ‘psych apps’• Can gather precise, objective,
sustained, ecologically valid data on real-world behaviors & experiences of millions
Smartphone research methods• “The smartphone psychology
manifesto” (2011), Perspectives on Psychological Science
• Symposia at:– Society for Experimental Social
Psychology (Austin Oct 2012)– Association for Psychological Science
(D.C., May 2013)– Association for Consumer Research
(Chicago Oct 2013)
What can smartphones do for psychology?
Example:Samsung Galaxy S3
Released June 2012Sold 20m within 3 months$1 bn/month in profits
Carried throughout day• Small: 5.4 x 2.8 x 0.3”• Light: 4.7 ounces• Reliable: 43 hour battery life• Continual data-gathering:
sensors, GPS, app logs, call logs
Familiar, intimate, personalized, trusted, unobtrusive• habituation low reactance• ecological validity
Samsung Galaxy S3
Samsung Galaxy S3Android 4.1 operating system
Develop & download “psych apps” for studies
Apps can run in background w/o annoying subjects
“Root access” to all hardware, sensors
msec timing unlike Mturk (e.g. Dufau 2011 lexical decision app N=4,157)
Software access to all other apps: phone calls, emails, web browser, Facebook
Samsung Galaxy S3
Samsung Galaxy S3pocket supercomputer
ARM Cortex A9 Chip with Exynos Quad-core 1.4 GHz CPU
Can run sophisticated AI, speech recognition, psych apps
21 Mbps 4G Broadband access cloud computing for unlimited processing & memory
Faster than your office PC
Samsung Galaxy S3personal data repository
2 GB RAM, 64 GB flash memory Medical records MRIs Genomes
School records Legal records Spending patterns Credit history
Contact lists Music tastes Photo albums Home videos
Samsung Galaxy S3Input/Output for running surveys, experiments
1280x720 pixel 4.8” touchscreen
Rear 8 MP camera (1080p@30fps video)
Front 1.9 MP camera (720p@30fps video)
Smart Stay eye tracking
Dual mikes, noise cancelling, voice recognition
Button taps, gestures, virtual keyboard
Samsung Galaxy S3Input/Output for high-quality audio & video
3.5mm stereo jack
microUSB2.0, MHL to HDMI to 3-D HDTV video & 8-channel digital audio
Samsung Galaxy S3Connectivity: download psych apps, upload data, sense environment & interactions
Bluetooth, WiFi: sense nearby electronics
4G broadband internet (HSPA+ @ 21 Mbit/s)
NFC payments, purchases
Samsung Galaxy S3Onboard sensors: location,
300 GPS + 24 GLONASS navsats: +/- 10m lat/long,10 nsec time
GISdigital maps
GPS receiver chip
life-tracking, spatial behavior, context-aware experiments
Samsung Galaxy S3Onboard sensors: movements
3-axis accelerometer: motion tracking, activity type, energy output, health, mood
3-axis gyroscope: orientation tracking Barometer: altitude,
weatherDigital compass: direction tracking
External sensors• Consumer EEG headsets, e.g,. Emotiv
EPOC 14-channel ($300)• Future biological microelectromechanical
systems (BioMEMS): wearable, implanted, injected to monitor temp, blood pressure, pulse, blood alcohol & drugs, hormones, immune system, inflammation, stress, ovulation, brain function
• Remote neuropsych, health psych, substance abuse research
Wireless intraocular glaucoma pressure sensor (1 cubic millimeter)
External biosensors:Continua Health Alliance
• Trade association of 220 companies • Promotes mHealth, certifies Bluetooth &
Zigbee biosensors – heart ECG, blood glucose monitors, ultrasound imagers
• Promoters: Samsung, Sharp, Panasonic, Sony, Fujitsu, Intel, IBM, Texas Instruments, Oracle, Cisco, Orange, Qualcomm, Novartis, Roche, UnitedHealth
mHealth = health care & research through mobile phones
• open-source mHealth specs, standards, software for:– Surveys, biosensors, RFID med tracking,
behavior tracking (GPS, sensors, call logs) – Continuous symptom monitoring integrated
across diseases – Whole-population drug trials, N-of-1
longitudinal trials
• Current projects: PTSD, pain management
• Want to work with psychologists: behavioral data sensing & standards
Deborah Estrin Ida Sim
PracticalitiesTechnical challenges• Conflicting apps• Limited battery power (esp. for GPS)• Limited-accuracy sensors (but getting better)• Heat dissipation (if psych app is CPU -hungry)• Unreliable telecoms service in some countriesParticipant behavior challenges• Forgetting to charge & carry smartphone daily• Losing or upgrading phones during study• Lending phones to others • Malicious hackers• Reactivity & self-consciousness
PracticalitiesGlobal recruitment potential, but constrained by
– Selection biases: who owns smartphones– Smartphone OS, drivers, hardware specs– Language of app interface – Geoareas covered by GIS databases– Payment systems, exchange rates
Programming psych apps• Software development kids (devkits) –
assume knowing Java, Objective-C, C#, etc.• No easy-to-use psych app devkits yet• Collaborate with computer scientists, user
experience researchers, smartphone manufacturers, telecoms service providers
Niko Kiukkonen et al. (2010) at Nokia: • Tracked 168 smartphone users 4 months eachData flood:• 15 million Bluetooth scans of nearby devices• 13 million Wi-Fi scans of nearby routers• 5 million GPS records• 4 million app usage records• 500,000 accelerometer readings• 220,000 audio samples• 130,000 voice calls• 90,000 text messages• 28,000 photos taken• 2,000 videos shot
Data analysis challenges
= data output from 70 participants recording one hour of HD video per day
1080p 30fps Pivothead recording glasses, $350
CERN LHC computing grid: 300 MB/s raw data output, handled by 200,000 processors and 150,000 TB disk space across 34 countries; 15 petabytes/year
Psych will have develop new ‘behavioral informatics’ methods
=
• Anonymity impossible given GPS, app usage, call logs, sensor logs
• Truly informed consent hard to get • Software licensing agreements rarely read –
Google, Facebook, Apple data-mining • Confidentiality vulnerable to authorities,
telecoms service providers & hardware manufacturers (e.g. Huawei)
• Rules unclear given global recruitment, access to vulnerable recruits, data storage with cloud computing
• Potential for liability, fraud, identity theft, blackmail
Human subjects & IRB challenges
Smartphones vs. other research methods
Pros:• Potential global recruitment & very large
samples• High convenience, ecological validity,
unobtrusiveness• Easy video/audio capture, motion sensing,
location tracking• Potential HD video/audio given common
peripherals• Potential remote neurophysiology & health
psych given Bluetooth biosensors
Smartphones vs. other research methods
Cons:• Substantial study prep work writing & field-
testing the psych app• Low contextual control over participant’s
social & physical environment• Potentially very large datasets requiring new
behavioral informatics methods• Ethical challenges in informed consent,
anonymity, confidentiality, liability, & rule ambiguity given global recruitment & cloud data storage
Challenges for Berkman?• Legal & moral issues: human subjects
protection, privacy, data security• Economic issues: payment systems• Social issues of behavioral Big Data &
personal data
Conclusions• A tech windfall for psychology• Telecoms R&D $billions creating
a pervasive, unified, global, context-aware system for sensing behavior & experience of almost everyone on earth
• Smartphones will quickly grow even more powerful & ubiquitous
• We just have to learn how to tap into the data-torrent
• Questions?
Extra slides for answering questions
Big Data in psychology
N=thousands
N > 100,000 (moral values: Jesse Graham et al., 2010)
N=61m (political influence: Robert Bond et al., 2012 , Nature)
YourMorals.org
Previous research using mobile electronic devices
• Analyze anonymized telecoms call-routing records: movements & social connections of millions of people
• Distribute preprogammed PDAs, EARs: experience sampling, diary studies, conversation samples
• Distribute preprogammed smartphones: MIT Reality Mining project
• Distribute apps remotely to existing smartphones: MyExperience, Mappiness– Recruit any user, anywhere– Huge samples– Rich data
Smartphones: all-purposeelectronic Swiss army knifes
• Replacing: landline phones, digital cameras, photo books, video recorders, MP3 music players, radios, voice recorders, GPS navigators, handheld game consoles, watches, alarm clocks, calendars, and calculators
• Can replace paper-and-pencil surveys, mail surveys, phone surveys
• Given the right peripherals, could replace many lab studies, field studies, and Internet studies
• RFID chip readers, can sense all tagged objects nearby (30 bn RFID tags produced per year; soon > 1 tr): consumer psych
• Better picoprojectors (e.g. Samsung Beam): group experiments
• Electronic glasses & contact lenses, augmented reality apps & experiments
• Better speech recognition• Bluetooth haptic, kinematic, mo-cap
peripherals (like Xbox 360 Kinect)• Piezoelectric clothing to charge batteries,
record joint movements
What will smartphones do soon?
Moore’s law: transistor density on chips (computer speed) doubles every two years
What will smartphones do soon?
By 2016: 8-core 2-GHz CPUs, 64-core GPU for 3D HD video
Smartphones as handheld supercomputers
1997: Intel ASCI Red, Sandia Nat’l Labs: first teraflop supercomputer9,298 333-MHz processors, 104 cabinets, 2,500 square feet, $50 million
Within 10 years: 64-core 50-GHz processors … plus unlimited cloud computing power, c. $500
=
Kryder’s law: memory storage capacity doubles every 12 months
16 GB micro SD flash memory cards
2012: 32 GB flash memory chip in Galaxy Nexus
2025: >200 TB memory in smartphones
Connectivity: Nielsen’s law: Internet connection speeds increase 50% per year
2012: HTC Thunderbolt = 300 Mbit/s LTE broadband
2025: 60 Gbit/s = 5x HDMI 1.3 cable = drive 5 HD digital cinema projectors
From phone-centric to data-centric devices
• 1987: ‘Wall Street’ phone• 1993: first smartphone IBM
‘Simon’ • 2007: Apple iPhone• 2008: Android OS
Evolution of the mobile phone1983 - 2012
Heat dissipation: Koomey’s law: computations per kWh energy dissipated increases 50% per year
Keep smartphones from getting too warm
Abstract
• 5.9 billion people now use mobile phones, of which 1.1 billion are smartphones. Smartphones will empower behavioral scientists to collect terabytes of ecologically valid data from vast global samples – easily, quickly, and remotely. Smartphones can record where people are, what they are doing, and what they can see and hear. They can run interactive surveys, tests, and experiments through touch screens and Bluetooth peripherals. This talk focuses on what smartphones can do now, and will be able to do in the near future, as research platforms. Smartphone research will require new skills in app development, Big Data analysis, and recruitment through social media, and will raise tough new ethical issues, but smartphones could transform the behavioral sciences even more profoundly than PCs and brain imaging did. By 2025, billions of potential research participants will be carrying ultra-broadband, sensor-rich smartphones with GPS, augmented reality goggles, and biosensors that allow remote psychophysiology. These will render some current research methods obsolete, and will open extraordinary new opportunities for understanding human nature and culture.