Sensing Posture-Aware Pen+Touch Interaction on Tablets

Explore innovative sensing techniques for posture-aware pen and touch interaction on tablets in this 20-minute conference talk from the ACM CHI Conference on Human Factors in Computing Systems. Discover how combining raw capacitance touchscreen images, inertial motion, and electric field sensors around the screen bezel enables adaptive interfaces that respond to user posture, hand grips, screen angle, and approach direction. Learn about posture-aware pen+touch techniques that morph UI elements to suit various contexts, including the Thumb Tool, Palm Tool, Fan Tool, and Smart Autorotation. Gain insights from the user study conducted and understand how these advancements can enhance the tablet user experience by adapting to fine-grained contexts of body-, arm-, hand-, and grip-centric frames of reference.