ETac: A Lightweight and Efficient Tactile Simulation Framework for Learning Dexterous Manipulation

Teaser image for the project

ETac: a lightweight and efficient tactile simulation framework. (a) ETac estimates elastomer surface deformations with fidelity comparable to that of FEM. (b) It enables large-area tactile sensing for dexterous manipulators while supporting large-scale RL training.

Abstract

Tactile sensors are increasingly integrated into dexterous robotic manipulators to enhance contact perception. However, learning manipulation policies that rely on tactile sensing remains challenging, primarily due to the trade-off between fidelity and computational cost of soft-body simulations. To address this, we present ETac, a tactile simulation framework that models elastomeric soft-body interactions with both high fidelity and efficiency. ETac distills knowledge from high-fidelity yet computationally expensive finite element simulations to a lightweight deformation propagation model, achieving simulation quality while enabling large-scale policy training. When serving as a soft-body simulation backend, ETac produces surface deformation estimates comparable to FEM and demonstrates applicability for creating digital twin of tactile sensors. Then, we showcase its capability in training a blind grasping policy that leverages large-area tactile feedback to manipulate diverse objects. Running on a single RTX 4090 GPU, ETac supports reinforcement learning across 4,096 parallel environments, achieving a total throughput of 869 FPS. The resulting policy reaches an average success rate of 84.45\% across four object types, underscoring ETac's potential to make tactile-based skill learning both efficient and scalable.

High Fidelity Deformation Estimation

We evaluated ETac's simulation quality by estimating sensor elastomer deformation. Indentation experiments are conducted on two elastomer types: a flat elastomer and a curved BioTac-style elastomer.



Comparison of elastomer surface deformation & displacement fields estimated by ETac and FEM. Arrows show node-wise displacement direction and magnitude, with color shifting to red as z-axis displacement increases.

ETac + Dexterous Manipulators: Large-area Tactile Sensing


During contact-rich interactions, ETac enables large-area tactile sensing simulation for dexterous manipulators.

Simulating Real Sensors

Next, we verified that the elastomer surface deformation simulated by ETac can be used to predict real sensor outputs.


We equipped tactile sensors (one with flat elastomer layer and another curved) in real and simulated settings, recording real sensor readings and surface deformations from ETac and FEM. A PointNet model was trained to map displacement fields to sensor responses.

Predicting signals of real sensors. Here we demonstrate the time series response of the most heavily loaded taxel and the outputs of all eight taxels at peak load. ``Unseen Trajectory'' denotes novel loading patterns with seen indenters.

RL Validation: Blind Grasp


We evaluated ETac's effectiveness for training reinforcement learning policies in a vision-deprived blind grasping task, whose objective is to manipulate and grasp a variety of objects according solely robot proprioception and tactile feedback.

Blind grasping process. The hand continuously adjusted its pose based on tactile feedback, gradually enclosed and ultimately lifted the object.