Vision, Touch & Sound for Robustness & Generalizability in Robotic Manipulation

Overview

Explore cutting-edge research on robotic manipulation skills in this Stanford University seminar. Delve into the challenges of high-dimensional state and action spaces, as well as sensor and motor control uncertainties. Discover how Assistant Professor Jennifer Bohg investigates representations of raw perceptual data to enhance robot learning and performance, particularly focusing on the integration of touch sensing with other modalities. Learn about crossmodal compensation models, corrupted sensor detection, and the application of deep learning and physically interpretable parameters in robotic manipulation. Examine innovative approaches like differentiable audio rendering and impact sound modeling for improved robot perception and control. Gain insights into end-to-end learning, robot source separation experiments, and key takeaways from the DiffImpact project, all aimed at advancing robustness and generalizability in robotic manipulation.

Syllabus

HAI Weekly Seminar
Previous work
Experimental setup
Learning generalizable representatie
Dynamics prediction from self-supervi
How is each modality used?
Overview of our method
Lessons Learned
Overview of today's talk
Related works
Crossmodal Compensation Model CC
Training CCM
Corrupted sensor detection during deploy
CCM Task Success Rates
Model-based methods fit physically interpretable parameters
Deep learning-based methods can lean from data in the wild
Differentiable audio rendering can learn interpretable parameters from data in the wild
Difflmpact gets the best of both worlds impact sounds
Physically interpretable parameters are easier to reuse
Decomposing an impact sound is an posed problem
Modeling rigid object impact forces
Parameterizing contact forces
Optimize an L1 loss on magnitude spectrograms
Analysis by Synthesis Experiment
Analysis by Synthesis: Ceramic Mug
End-to-End Learning ASMR: Ceramic Plate
Robot Source Separation Experiment
Steel Fork and Ceramic Mug
Difflmpact's Key Takeaways
Conclusions
Thank you for your Attention