Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer InteractionTobias Grosse-Puppendahl, Christian Holz, Gabe Cohn, Raphael Wimmer, Oskar Bechtold, Steve Hodges, Matthew S. Reynolds, Joshua R. Smith
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
Frank Beck and Bent Stumpe: Two devices for operator interaction in the central control of the new CERN accelerator
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensing apacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensingapacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensing apacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
A New Taxonomy
Research Challenges
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
Proposed Taxonomy
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
Proposed Taxonomy
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
Proposed Taxonomy
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensing apacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
A New Taxonomy
Research Challenges
Crawling > 5900 Papers indexed by IEEE and ACM 1.
Crawling > 5900 Papers indexed by IEEE and ACM 1.
Review of each paper: Does it match our criteria? 2.
Tagging of 193 relevant papers by > 2 authors3.
Crawling > 5900 Papers indexed by IEEE and ACM 1.
Review of each paper: Does it match our criteria? 2.
[Video of Tagging Process]
0
20
40
60
80
100
120
140
NovelApplications of
CapacitiveSensing
HardwareContribution
QuantitativeStudy
Algorithm Model ofElectrical
Properties
Toolkits forCapacitive
Sensing
SimulationResults
Type of Contribution (193 Papers)Read our paper: aka.ms/capsense
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensing apacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
A New Taxonomy
Research Challenges
Activ
e Ca
paci
tive
Sens
ing
Shunt Mode
A Taxonomy for Capacitive Sensing
Thomas G. Zimmerman, Joshua R. Smith, Joseph A. Paradiso, David Allport, and Neil Gershenfeld. 1995. Applying Electric Field Sensing to Human-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95).
Activ
e Ca
paci
tive
Sens
ing
Shunt Mode
A Taxonomy for Capacitive Sensing
Thomas G. Zimmerman, Joshua R. Smith, Joseph A. Paradiso, David Allport, and Neil Gershenfeld. 1995. Applying Electric Field Sensing to Human-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95).
Jun Rekimoto: SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. CHI ‘02.
SK Lee, William Buxton, and K. C. Smith. 1985. A multi-touch three dimensional touch-sensitive tablet. CHI’85.
Activ
e Ca
paci
tive
Sens
ing
Shunt Mode
A Taxonomy for Capacitive Sensing
Thomas G. Zimmerman, Joshua R. Smith, Joseph A. Paradiso, David Allport, and Neil Gershenfeld. 1995. Applying Electric Field Sensing to Human-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95).
Joshua R. Smith: Electric Field Imaging (PhD Thesis). 1999.
Jun Rekimoto: SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. CHI ‘02.
SK Lee, William Buxton, and K. C. Smith. 1985. A multi-touch three dimensional touch-sensitive tablet. CHI’85.
Activ
e
Shunt Mode
A Taxonomy for Capacitive Sensing
Transmit Mode Receive Mode
Activ
e
Shunt Mode
A Taxonomy for Capacitive Sensing
Transmit Mode Receive Mode
Paul Dietz and Darren Leigh. 2001. DiamondTouch: a multi-user touch technology. UIST ’01.
Activ
e
Shunt Mode
A Taxonomy for Capacitive Sensing
Transmit Mode Loading ModeReceive Mode
Activ
e
Shunt Mode
A Taxonomy for Capacitive Sensing
Transmit Mode Loading ModeReceive Mode
Jun Rekimoto, Takaaki Ishizawa, Carsten Schwesig, and Haruo Oba. 2003. PreSense: interaction techniques for finger sensing input devices. UIST ‘03.
Chia-Hsun Lee and Ted Selker. 2004. iSphere: a proximity-based 3D input device. In SIGGRAPH ‘04
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
H. Prance, P. Watson, R. J. Prance, and S. T. Beardsmore-Rust. 2012. Position and movement sensing at metre standoff distances using ambient electric field. Measurement Science and Technology 23, 11 (2012), 115101.
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
Rekimoto et al.: Sensing GamePad: electrostatic potential sensing for enhancing entertainment oriented interactions. CHI ‘04
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
A Taxonomy for Capacitive Sensing
Gabe Cohn, Daniel Morris, Shwetak Patel, and Desney Tan. Humantenna: Sensing Gestures Using the Body as an Antenna. In CHI ’12.
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
A Taxonomy for Capacitive Sensing Read our paper: aka.ms/capsense
capacitive sensor capacitive sensing mutual capacitance sensing Capacitively coupled body mmunication capacitive touch sensor capacitive touch sensing Electric Potential Sensing PowerLinsitioning Loading mode electric field sensing Capacitive Proximity Sensing Electrostatic Inductio ctric Field Imaging n/a Measurement of Electric Field bioimpedance Passive Sensing of Changes an Body Electric Potential capacitive multi-touch sensing electro-magnetic sensing Ambient elect
eld sensing TouchNet Intrabody Communication Intrabody communications Mutual-Capacitance sing AC hum detection combined inductive-conductive proximity sensing carrier-frequency metho
ear field imaging Body-coupled communications capacitive coupling capacitance proximity sensing body area communication touch-identification tokens Self-Capacitance capacitive sensor arrays capacitance measuring compressive sensing on capacitive touch screens active capacitive sensing apacitive positioning capacitance sensor intra-body communication human body communication pacitive finger sensing 2.5D capacitive touch sensor capacitive sensor array capacitive multitouchensor electric field ranging projected capacitance capacitive touch panel Electrostatic Potential ensing capacitive touch communication time-multiplexed loading mode capacitive sensing Swept Frequency Capacitive Sensing Capacitive widgets indirect capacitive sensing using electromagnetic rference single capacitive touch sensor capacitive touch-sensing EM sensing capacitive coupling
low-frequency signals electromagnetic noise sensing electromagnetic interference capacitive teractions capacitive charging time passive measurement of static electric field of the environment
acitive and resistive touch sensing static electric field sensing capacitive position measurements sk potential level Swept Frequency Capacitive Sensing Capacitive Near Field Communication
A New Taxonomy
Research Challenges
Research Challenge #1: Sensitivity to Grounding
How can we bring these powerful interactions to devices that only have floating references for practical applications?
Research Challenge #1: Sensitivity to Grounding
Research Challenge #1: Sensitivity to Grounding
Masaaki Fukumoto and Mitsuru Shinagawa.CarpetLAN: A Novel Indoor Wireless(-like)Networking and Positioning System. Ubicomp ‘05
Electro-optic crystals:
A. Pouryazdan, R. J. Prance, H. Prance, and D. Roggen. 2016. Wearable electric potential sensing: a new modality sensing hair touch and restless leg movement. UbiComp ‘16
Ultra-high input impedance sensors:
Valkyrie Savage et al.: Midas: fabricating custom capacitive touch sensors to prototype interactive objects. UIST ’12.
Cutting Ink-based Printing
Nan-Wei Gong et al.: PrintSense: a versatile sensing technique to support multimodal flexible surface interaction. CHI ’14.
3D Printing
Martin Schmitz et al.: Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects. UIST ’15
Research Challenge #2: Support for End-to-End Prototyping
Valkyrie Savage et al.: Midas: fabricating custom capacitive touch sensors to prototype interactive objects. UIST ’12.
Cutting Ink-based Printing
Nan-Wei Gong et al.: PrintSense: a versatile sensing technique to support multimodal flexible surface interaction. CHI ’14.
3D Printing
Martin Schmitz et al.: Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects. UIST ’15
We are making great advances with prototyping electrodes….But real end-to-end systems are still hard to prototype.
Research Challenge #2: Support for End-to-End Prototyping
Hsin-Liu (Cindy) Kao, Artem Dementyev, Joseph A.Paradiso, and Chris Schmandt. 2015. NailO: NailO: Fingernails as an Input Surface. CHI ’15.
Research Challenge #2: Support for End-to-End Prototyping
Hsin-Liu (Cindy) Kao, Artem Dementyev, Joseph A.Paradiso, and Chris Schmandt. 2015. NailO: NailO: Fingernails as an Input Surface. CHI ’15.
Super small and thin prototyping electronicsfor capacitive sensing are needed.
Research Challenge #2: Support for End-to-End Prototyping
Research Challenge #2: Support for End-to-End Prototyping
1. Reduce the size of prototypingelectronics
2. Reduce the size of the battery by usinghybrid active / passive capacitive sensing
Steve Hodges, Nicolas Villar, Nicholas Chen, Tushar Chugh, Jie Qi, Diana Nowacka, and Yoshihiro Kawahara. Circuit stickers: peel-and-stick construction of interactive electronic prototypes. CHI ’14.
Nan-Wei Gong, Steve Hodges, and Joseph A. Paradiso. Leveraging conductive inkjet technology to build a scalable and versatile surface for ubiquitous sensing. UbiComp ’11.
Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, Mustafa Emre Karagozler, Carsten Schwesig, and Karen E. Robinson. Project Jacquard: Interactive Digital Textiles at Scale. CHI ‘16.
Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing.
Research Challenge #3: Enabling Flexible and Stretchable Applications
Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, Mustafa Emre Karagozler, Carsten Schwesig, and Karen E. Robinson. Project Jacquard: Interactive Digital Textiles at Scale. CHI ‘16.
Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing.
How can we make touch and gesture recognition robust?
Research Challenge #3: Enabling Flexible and Stretchable Applications
Liwei Chan, Stefanie Mueller, Anne Roudaut, and Patrick Baudisch. CapStones and ZebraWidgets. CHI ’12.
Tam Vu, Akash Baid, Simon Gao, Marco Gruteser, Richard Howard, Janne Lindqvist, Predrag Spasojevic, and Jeffrey Walling. Distinguishing users with capacitive touch communication. MobiCom 2012.
Passive Tags Active Tags
Research Challenge #4: Enriching Sensing with Communications
Liwei Chan, Stefanie Mueller, Anne Roudaut, and Patrick Baudisch. CapStones and ZebraWidgets. CHI ’12.
Tam Vu, Akash Baid, Simon Gao, Marco Gruteser, Richard Howard, Janne Lindqvist, Predrag Spasojevic, and Jeffrey Walling. Distinguishing users with capacitive touch communication. MobiCom 2012.
Passive Tags Active Tags
Higher spatial and temporal resolution touch-screens would make a difference.
Research Challenge #4: Enriching Sensing with Communications
Mehrdad Hessar, Vikram Iyer, and Shyamnath Gollakota. 2016. Enabling On-body Transmissions with Commodity Devices. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16).
Christian Holz and Marius Knaust. 2015. Biometric Touch Sensing. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST ’15.
Receive Transmit
Research Challenge #4: Enriching Sensing with Communications
Mehrdad Hessar, Vikram Iyer, and Shyamnath Gollakota. 2016. Enabling On-body Transmissions with Commodity Devices. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16).
Christian Holz and Marius Knaust. 2015. Biometric Touch Sensing. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST ’15.
Receive Transmit
Commercially available touchscreens impose limits on communication mechanisms
Research Challenge #4: Enriching Sensing with Communications
Research Challenges: Touch Screen Research @ CHI
Jun Rekimoto. SmartSkin: an infrastructure forfreehand manipulation on interactive surfaces. CHI ’02.
Darren Leigh, Clifton Forlines, Ricardo Jota, Steven Sanders, and Daniel Wigdor. High Rate, Low-latency Multi-touch Sensing with Simultaneous Orthogonal Multiplexing. UIST ‘14
Research Challenges: Touch Screen Research @ CHI
Jun Rekimoto. SmartSkin: an infrastructure forfreehand manipulation on interactive surfaces. CHI ’02.
Darren Leigh, Clifton Forlines, Ricardo Jota, Steven Sanders, and Daniel Wigdor. High Rate, Low-latency Multi-touch Sensing with Simultaneous Orthogonal Multiplexing. UIST ‘14
Reviving touch screen research can enable great new tangible and wearable experiences.
Activ
ePa
ssiv
e
Shunt Mode Transmit Mode Loading ModeReceive Mode
Not applicable
Conclusion Read our paper: aka.ms/capsense
Capacitance is Everywhere!
Research Challenges
• Towards Better Reproducibility• Sensitivity to Grounding• Implications for Real-World Deployments• Support for End-to-End Prototyping• Reducing Form Factor & Instrumentation• Enabling Flexible and Stretchable Applications• Unifying Approaches to Interpret Electric Fields• Enriching Sensing with Communications