fbpx

Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation

Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation

arXiv:2405.08576v1 Announce Type: new Abstract: Although pre-training on a considerable amount of information is useful for robotic studying, present paradigms solely carry out large-scale pretraining for visible representations, whereas representations for different modalities are educated from scratch. In distinction to the abundance of visible information, it’s unclear what related internet-scale information could also be used for pretraining different modalities corresponding to tactile sensing. Such pretraining turns into more and more essential within the low-data regimes widespread in robotics functions. In this paper, we tackle this hole by utilizing contact microphones instead tactile sensor. Our key perception is that contact microphones seize inherently audio-based info, permitting us to leverage large-scale audio-visual pretraining to acquire representations that increase the efficiency of robotic manipulation. To the most effective of our data, our methodology is the primary strategy leveraging large-scale multisensory pre-training for robotic manipulation. For supplementary info together with movies of actual robotic experiments, please see https://ift.tt/mKk5T4s.

HI-FI News

by way of cs.RO updates on arXiv.org https://ift.tt/FTks806

May 15, 2024 at 09:34AM

Privacy Preference Center

Necessary

This cookie is set by Google Analytics. It stores and update a unique value for each page visited and is used to count and track pageviews.

_gid , _ga

Advertising

Used by Meta to deliver a series of advertisement products such as real time bidding from third-party advertisers

_fbp

Analytics

This cookie is set by Google Analytics. It stores and updates a unique value for each page visited and is used to count and track pageviews.

_gid , _ga

Other

Tracks when someone clicks through a Klaviyo email to your website

__kla_id

Select your currency