ShadowTac: Dense Measurement of Shear and Normal Deformation of a Tactile Membrane from Colored Shadows

To robustly handle objects, robots must perceive mechanical interactions through touch with sufficient richness. New tactile sensors leverage miniature cameras to provide dense measurements of these interactions, allowing for the extraction of material properties and frictional information. Among the plethora of solutions, retrographic sensing is popular for its ability to finely resolve the shape of the object being touched. These sensors use a reflective membrane, illuminated at a shallow angle by three RGB lights from which fine details of the surface can be recovered. However, these retrographic sensors are unable to detect the lateral displacement of the membrane and, therefore overlook frictional information, which is crucial for grasping and manipulation. Embedding and tracking opaque markers has been a makeshift solution, but these markers occlude the membrane and are difficult to manufacture. In this paper, we introduce ShadowTac, a tactile sensor that combines retrographic illumination with non-intrusive markers created by colored shadows. We patterned the retrographic surface with a dense array of submillimeter dimples, which are small enough not to obstruct the view yet cast shadows large enough to be visible to the camera. ShadowTac captures a dense image of both the normal displacement field with fine details and a precise lateral displacement field by tracking the markers. Additionally, our sensor is easy to manufacture, as the dimple pattern can simply be molded. We evaluated the measurement reliability of ShadowTac and its effectiveness in estimating the incipient slip of arbitrary objects. The dense measurement of both the normal and shear deformation that the sensor captures makes it ideal for tracking dynamic interactions between robotic fingertips and manipulated objects.