Date post: | 15-Sep-2015 |
Category: |
Documents |
Upload: | phamvantuong |
View: | 223 times |
Download: | 3 times |
Android 2012: Integrating Sensors Into Android Hardware Rian Sanderson
Sensor Platforms Inc.
PresenterPresentation NotesNotes: * Rian Sanderson * share some of what Ive learned over the past year working with Android
* presentation is not about using sensors in an android app * not how transformative sensors are
* You have a board running android, you have senors on some daughter card * Theoretically there is open source code to bridge the gap between them * This is the presentation for you
Next Step: * Thats where I was a year ago
integrating android sensor
hardware
the source code is out there but not much of the big picture
the problem
google this
PresenterPresentation NotesNotes: * it wasnt great three months ago, it was worse a year ago, is now getting better * adding in a board name: beagle, snoball helps * just not much in the way of guides out there for people developing their own hardware * One of the best links: tells you how to build it, tells you to read a header file, but thats it * Another is a great guide from TI, I have a link to it on a further reading section
Next Step: * like many things in this business * if know what to look for more a matter of cranking it out
this talk should enable you to
integrate new sensors into Android debug each of the layers in the sensor stack
download these slides and check out the speakers notes if youd like to read all my bullet points
PresenterPresentation NotesNotes: * I'll show you what to look for and where * Introduce you to some tools to make sure you're getting it right * Illustrate the low level sensor architecture in Android * Hands on demo of tools for debugging at each layer * Based on Gingerbread release, but not substantially different in Ice Cream Sandwich
Next Step: * before diving in, take a step back
creating sensors systems is an art
Software System Integration Physical Conventions Sensor Fusion Hardware System Engineering Processor Resources
todays talk
the software integration is a craft that can be picked up quickly
PresenterPresentation NotesNotes:Sensor system design is more akin to RF design than traditional digital design
Focusing on Software integration today * only talking about 1/3 the software stack! * have a good diagram about this on Big Picture slide
Physical conventions * This is the stuff that rockets blow up over * acceleration in m/s^2, magnetometer in uT, accelerometer sees gravity as a force pointing UP * all the sensors aligned correctly on the board
Sensor Fusion * serious math * kalman filters * signal processing
Hardware System Engineering * Kevin Shaw speaking in Sensors Theatre on Thursday * garbage in garbage out * sensor noise * PCB flex corrupting accelerometer * magnetometer next to battery
Processor Resources * break this out because its so important * after cost, cost, and cost * responsiveness and battery life are arguably the next most important attribute of a great product * CPU choice, RAM, storage drive costs * goal is to keep that processor in power save * every instruction executed counts, especially at 100Hz
Next Step: * I want to introduce you the platform for todays demos
demo: BeagleTab
you too can integrate all the sensors you want into Android
PresenterPresentation NotesNotes: * Something only an engineer could love * resistive touch display, buttons * beagle board,batteries * sensor board: ** (Accel + Mag + Gyro) x3 ** breakout board + GPS + pressure + temp ** i2c + SPI through EXT connector * Sensor Fusion is done with FreeMotion library
Next Step: * This is doable, and if you have android up and running already, can get sensors in there very quickly
big picture
insert your code here
PresenterPresentation NotesNotes:sensors at the bottom, phone/tablet applications at the top
* Jim Steele is talking about how to develop apps that talk to sensors * Karim Youngers new book talks about android internals * android specifies a sensors hardware abstraction * libsensor aka sensors.so and is specific to android systems * you must implement sensors.so * huge amount of linux infrastructure to utilize in completing that task
Next Step:Will bounce around this diagram, in the manner of how I approach debugging a new Android board
data flow
PresenterPresentation NotesNotes: * drivers are loaded at boot time * Android Sensor Service starts and reads what sensors are available * App starts, requests sensor data from SensorManager * Sensor Service asks libsensor to go start up a sensor if its not already started * libsensor flips the switch in the driver, sensor starts up, starts spewing data * libsensor reads that data, reformats it for android, puts it in a queue * sensor service comes along and gets that data at a regular rate * delivers to each app subscribed to sensor manager
Next Step: * lets see one of those apps
app level debugging
AndroSensor displays sensor info exactly as an app
developer will see it
PresenterPresentation NotesNotes: * demonstrates the power and joy of android * I didn't have to spend time writing this * I know this works and is tested * shows exactly the information an app sees when querying Android.SensorManager
Next Step:skipping over the middle of the big picture, diving into the bottom
sensor drivers
Tools for debug
adb shell lsmod | grep
dmesg
dont write sensors drivers keep them platform agnostic
PresenterPresentation NotesNotes: * first thing with a new board, I want to see if drivers are working * Linux drivers are available for most any sensor out there * dmesg: is it working? * lsmod | grep : is it loaded * adb shell with a pipe -> utilize text processing of your dev machine
Next Step: * Youre probably compiling lots of drivers, so glossing over this
linux infrastructure
leverage the most
appropriate infrastructure
PresenterPresentation NotesNotes: * in software we talk about toolboxes, I think of linux as a warehouse * hard to find the most appropriate tools * best to build on what other people have done
Next Step: * highlighted input event framework because its so prevalent
Input Event Framework presents each sensor as name and a stream of data events
PresenterPresentation NotesNotes: * Historically for connecting mice and keyboards to X server * helps unify disparate driver interfaces * find right drivers, can process data from a OneWire thermo like an SPI accel * standard methodology, standard handling, standard tools, easy to debug
Next Step: * it unifies driver interface through event based processing
Input Event Processing
struct input_event { struct timeval time; __u16 type; __u16 code; __s32 value; };
// Event Types #define EV_SYN 0x00 #define EV_KEY 0x01 #define EV_REL 0x02 #define EV_ABS 0x03 ... // Event Codes #define REL_X 0x00 #define REL_Y 0x01 #define REL_Z 0x02 ...
#include
4 event structs per sensor measurement uses code field to differentiate axis
PresenterPresentation NotesNotes: * recommend reading the header
* Types: ** are a little mismatched for sensor data ** EV_ABS is for absolute measurements- think Touchpad ** EV_REL is for relative measurements- think mouse ** EV_SYN is a sync record to say that These events all go together
* Codes: ** code tells you the axis
Next Slide: * see this in action with a demo
Demo: Input Event Framework
PresenterPresentation NotesNotes:Hands on demo at command line using * cat /proc/bus/input/devices * getevent /dev/input/eventX
Next Slide: * back to the presentation
Input Event Framework Resources
tools for debug
getevent /dev/input/event cat /proc/bus/input/devices
#include
further reading http://www.kernel.org/doc/Documentation/input/ http://www.kernel.org/doc/Documentation/input/event-codes.txt http://en.wikipedia.org/wiki/Evdev Internal input event handling in the Linux kernel and the Android userspace
PresenterPresentation NotesNotes: * input.h is an approachable header to read * get past mouse and keyboard focus * check out the links under further reading * standard tools, standard debug
Gotchas: * beyond x,y,z standards arent so standard * what if you want to return something like calibration data? * EV_ABS events can get dropped by framework * EV_REL events will get dropped if exactly the same as previous event
Next Step: * just showed how raw sensor data comes into the system, but how does android get a hold of it?
libsensor advertises available sensors and makes them available to
Sensor Service
controls sensors and reads data using Linux infrastructure
PresenterPresentation NotesNotes: * Sensor Service asks for things * libsensor handles these requests using the linux infrastructure
Next Step: * the first request: what sensors does this board have?
libsensor advertises sensors
struct sensors_module_t { struct hw_module_t common; int (*get_sensors_list)(struct sensors_module_t* module, struct sensor_t const** list); };
libsensor must define a sensors_module_t struct named
HAL_MODULE_INFO_SYM has a func ptr which returns a list of sensor_t strycts describing
the sensors
PresenterPresentation NotesNotes: * not immediately obvious from the header what to do * define HAL_MODULE_INFO_SYM * object oriented C with function pointers in the structs * comments removed here, and they help a little * will look at how this is commonly implemented later and it will be crystal clear
Next Step: * libsensors makes sensors available
libsensor makes sensors available
struct sensors_control_device_t { struct hw_device_t common; int (*activate)(struct sensors_control_device_t *dev, int handle, int enabled); int (*set_delay)(struct sensors_control_device_t *dev, int32_t ms); int (*wake)(struct sensors_control_device_t *dev); }; struct sensors_data_device_t { struct hw_device_t common; int (*data_open)(struct sensors_data_device_t *dev,
native_handle_t* nh); int (*data_close)(struct sensors_data_device_t *dev); int (*poll)(struct sensors_data_device_t *dev, sensors_data_t* data);
object oriented interfaces to
control and poll sensors
PresenterPresentation NotesNotes: * turning on/off , getting data from them * putting function pointers in structs * comments are removed to focus on methods * method names in blue * for each sensor, you declare one of each of these structs * is much less opaque when you see code that implements it
Next Step: * this is how data is made available to android, but what about data formats?
libsensor reads sensor drivers
typedef struct { union { float v[3]; struct { float x; float y; float z; }; ...
translate from
4 input_events to 1 sensors_vect_t
struct input_event { struct timeval
time; __u16 type; __u16 code; __s32 value; };
put into physical units: deg C, m/s^2
PresenterPresentation NotesNotes: * Sensor Manager asked for data, but need to go out to drivers and get it * input event framework is an easy way to get at our sensor data * drivers have a fixed point representation * android wants data in SI units: m/s^2, degrees Celsius,
Next Step: * how are we doing? Starting to make sense?
libsensor
advertises available sensors and makes them available to Sensor Service
controls sensors and reads data using Linux infrastructure
#include
PresenterPresentation NotesNotes: * recap of what libsensors does * libsensor interface in hardware/sensors.h * not to be confused with any other sensors.h, or linux lm_sensors
Next Step: * how do you actually implement those functions?
adapt an existing implementation
rowboat / hardware-ti-omap3 / rowboat-gingerbread / libsensors / sensors.cpp
rowboat / hardware-libhardware / include / hardware / sensors.h
this is the most common implementation, and is extendable for input event drivers
PresenterPresentation NotesNotes: * open both files, start in sensors.h * talked about input event framework because its the basis for the most common implementation * notice the helpful introductory comments on physical conventions * browse to sensors_module_t, which is implemented in sensors.cpp as sSensorList * later on in the file
Next Step: * rest of Sensors.cpp starts to rely on SensorBase classes to do the control and data structs
root of the implementation
best thing about it is that it works
PresenterPresentation NotesNotes: * public methods mirror a device driver * private methods for dealing with file handles, opening, closing * Best thing about it is that it works, extendable via copy-paste * Im not a fan, because encourages copy/paste * an example of why people dont like C++, but not the fault of C++
Next Step: * must derive a concrete class to utilize it
SensorBase and friends rowboat / hardware-ti-omap3 / rowboat-gingerbread / libsensors / SensorBase.h / AccelSensor.h / GyroSensor.h / InputEventReade
copy/paste if you
have an input event driver
PresenterPresentation NotesNotes: * example from TI repo with Accel and Gyro * much of the functionality is in sensor base * copy paste from the concrete classes AccelSensor GyroSensor * concrete classes have details like device name * InputEventCircularReader does the hard work of converting the input events into sensor_vec_t structs
Next Step: * encourage you do download this source code
libsensor
further reading http://www.kandroid.org/online-pdk/guide/sensors.html
source code
root / device / samsung / crespo / libsensors rowboat / hardware-ti-omap3 / rowboat-gingerbread / libsensors
OpenEtna / android_device_lg_eve / android_device_lg_eve / libsensors
PresenterPresentation NotesNotes: * here are three implementations of libsensor * TI BeagleBoard * Samsung Nexus One aka Crespo * an alternative, somewhat older, libsensors implementation on a phone Ive never heard of * Please share any libsensor specific documentation links you find with me * link from kandroid talks about building
Next Step: * more links..
Linux Infrastructure links
further reading Android Sensor PortingGuide Linux Industrial I/O Subsystem
kernel.org/.../spi/spi-summary kernel.org/.../i2c/summary
kernel.org/.../w1/w1.generic
Getting Started With UInput
PresenterPresentation NotesFurther Reading * recently released, hard to find guide from TI covers similar info as I have * IIO is an alternative to Input Framework, targeted towards high speed A/D drivers and Sensor drivers * Kernel documentation- surprisingly well written, not mired in the details * uinput for making virtual input devices, simulated sensor data
Next Step: * to wrap it all up
remember
keep drivers platform agnostic
leverage existing Linux infrastructure
implement the glue between Android and Linux
PresenterPresentation NotesNotes: * platform specific data goes into lib sensors
Gotchas:be cognizant of GPL and Android Open Source licensing issuesdefinitely GPL at the kernel levellikely AOSL in userland
Next Slide * what I didnt talk about thats going to impact you
whats next
Ice Cream Sandwich not that different Non input-framework based drivers Sensor Fusion daemons
now
Open Sensor Processing standards Dedicated Sensor Processors Sensor Fusion going beyond just orientation
future
PresenterPresentation NotesNotes: * didnt get time to talk about these things * happy to answer questions on them
Ice Cream SandwichAPIs largely the samesome sensor fusion includedEmerging Sensor Related Standards WC3StreamInputDedicated Sensor ProcessorsSensor HubsBig/Little processorsAlternative sensor configurations * wireless sensors * USB based sensors
Alternative driver frameworks * non-input event based drivers * io-ctl based sampling drivers * Linux IIO for more efficientSensor Fusion Daemons * Platform Testing and Verification * using uinput to simulate data
Questions?
Slide Number 1the problemthis talk should enable you tocreating sensors systems is an artdemo: BeagleTabbig picturedata flowapp level debugging sensor driverslinux infrastructureInput Event FrameworkInput Event ProcessingDemo: Input Event FrameworkInput Event Framework Resourceslibsensorlibsensor advertises sensors libsensor makes sensors availablelibsensor reads sensor drivers libsensoradapt an existing implementationroot of the implementationSensorBase and friendslibsensor Linux Infrastructure linksrememberwhats nextQuestions?