Open Source Smartphone Libraries for Computational Social Science
@neal_lathia, k. rachuri, c. mascolo, g. roussosseptember 2013
“...each of these transactions leaves digital traces that can be compiled into comprehensive pictures of both individual and group behaviour...
“Computational Social Science” Lazer et. al
AccelerometerMicrophoneCameraGPSCompassGyroscopeWi-FiBluetoothProximityNFCLight
“... a number of challenges remain in the development of sensor-based applications [...] there is mixed API and operating system (OS) support to access the low-level sensors...”
“A Survey of Mobile Phone Sensing,” Lane et. al
● Battery-friendly sensor data collection● Triggering notifications● Data storage & transmission
“Reinventing the Wheel”
All smartphone-based research needs to begin by engineering solutions for:
if you don't like code, turn away now
● Current approach:– Requires treating each sensor with different code– Hides battery & energy efficiency requirements
● We have built Android ESSensorManager– Everything is a “sensor”– Simple API with two modes (get, subscribe)– API exposes battery issues to programmer
Sensor Data Collection
● Pull Sensors– Accelerometer, Location, Microphone– Wi-Fi, Bluetooth, Camera– Active apps, SMS/Call Log Content
● Push Sensors– Battery, Connection State
– Proximity, Screen– Phone Calls/SMS Events
Everything as a 'Sensor'
aim: get data in 2 lines of code.
// get the instance
ESSensorManager sm = ESSensorManager.getSensorManager(context)
// ask for some data
MicrophoneData data
= (MicrophoneData) sm.getDatafromSensor(
SensorUtils.SENSOR_TYPE_MICROPHONE)
Get data, fast
aim: quickly configure sensing & respond to battery
// make a subscription
int sid = sm.subscribeToSensorData(
SensorUtils.SENSOR_TYPE_MICROPHONE, listener)
// deal with data pushed to you
class Listener implements SensorDataListener
{
public void onDataSensed(SensorData d){..}
public void onCrossingLowBatteryThreshold(..) {..}
}
Get data, continuously
● “I need to ask the user for some interaction...”– Based on time of day, randomly– Based on a sensor event– ESTriggerManager
● “I need the data to come back to me!”– Without using up all of my participant's 3G
connection– ESDataManager
Problems #2 & #3
● JSON formatting– Flexible for various sensors
– Includes sensing configuration{
"zAxis":[9.959879,9.959879,....],
"senseStartTime":"17:07:00:281 16 05 2013 -0500 CDT",
"sampleLengthMillis":8000,
"xAxis":[0.11492168,0.11492168,0.0766144548,...],
"yAxis":[0.11492168,0.11492168,0.0766144548,...],
"sensorTimeStamps":[1368742020298,1368742020488,....],
"sensorType":"Accelerometer"
}
Data Management
● Simple on-phone data querying– Only queries local data
– “Give me all the accelerometer data from the last hour...”
Data Management
● Configurable Transfer Policy– Do not transfer (local storage only)
– Transfer immediately (or fail)
– Asynchronous transfer (Wi-Fi & timeout)
Data Management
● Student Project at Birkbeck College, London– Post-graduate students with programming
experience
– Sample Audio data from the environment– Measure noise pressure/audio features– Post data to COSM (Xively)
Preliminary Feedback
● Why wasn't it used?– Lack of experience; support for emulator
● Using the library– Easy, quick
– Substantially less code (~ better quality): focus is on app features, not sensor engineering
● What is missing?– Data filters; inference algos; simulated data
Preliminary Feedback
“it is useful for the research community to think about and propose sensing abstractions and APIs ...”
“A Survey of Mobile Phone Sensing,” Lane et. al
Open Source Smartphone Libraries for Computational Social Science
http://emotionsense.orghttps://github.com/nlathia/SensorManager https://github.com/nlathia/TriggerManager https://github.com/nlathia/SensorDataManager