Glonce

TouchScale pairs 500 hours of human video with touch data

researchThe Glonce editorsHugging Face🔥63Glonce rating63Impact55Novelty65Relevance90Surprise40Players15Credibility85
TouchScale pairs 500 hours of human video with touch data

A paper on Hugging Face introduces TouchScale, a 500-hour dataset of contact-rich human interaction recorded with a single unified wearable setup. The authors start from a gap. Large-scale egocentric human video is becoming an important source of physical supervision for embodied learning, but video alone leaves contact and pressure unrecorded. Recent visual-tactile datasets supply that missing signal, yet their synchronized tactile data remain far smaller in volume than human video. On top of that, the largest resources often merge recordings from different sensors or annotation procedures, which makes the effect of data scale difficult to isolate.

TouchScale is built to address both problems. Its approximately 2K predefined task descriptions span everyday activities and structured manipulation. Each recording temporally aligns egocentric RGB-D video with wrist RGB video and dense full-hand bimanual tactile measurements.

The authors report three kinds of results. First, compared with prior tactile data, training on the full TouchScale raises zero-shot contact IoU on data from an unseen tactile sensor from 0.134 to 0.383. Second, pretraining a visual encoder on TouchScale yields the highest action recognition accuracy on three benchmarks among the compared visual-tactile datasets. Third, when TouchScale is used for visual-tactile mid-training of a robot policy, the average real-world success rate across four contact-rich manipulation tasks goes from 22.5% to 57.5%, a gain of 35 percentage points.

With the sensor and collection protocol held fixed, both zero-shot tactile prediction and robot success show an overall upward trend as more TouchScale data is used. The authors say these results suggest that human visual-tactile data collected at scale with consistent sensing benefits both perception and robot manipulation. They state that they will publicly release TouchScale, including all synchronized visual-tactile recordings and reconstructed object models, to support future research on scalable visual-tactile learning.

Key facts

  • TouchScale is a 500-hour dataset of contact-rich human interaction recorded with a single unified wearable setup, with approximately 2K predefined task descriptions.
  • Each recording aligns egocentric RGB-D video, wrist RGB video and dense full-hand bimanual tactile measurements.
  • Training on the full dataset raises zero-shot contact IoU on an unseen tactile sensor from 0.134 to 0.383.
  • Visual-tactile mid-training of a robot policy on TouchScale lifts average real-world success across four contact-rich tasks from 22.5% to 57.5%.
  • The authors say they will publicly release the recordings and reconstructed object models.

Why it matters

Robot learning increasingly draws on egocentric human video, but video does not record contact or pressure, which define physical interaction. Existing visual-tactile datasets add that signal, yet their synchronized tactile data remain far smaller than human video, and the largest ones merge different sensors or annotation procedures. That mix makes it hard to tell what data scale alone contributes. TouchScale keeps one wearable setup and one collection protocol across 500 hours, so the effect of scale can be read more cleanly.

Who it affects

Researchers working on embodied learning, tactile perception and robot manipulation are the intended audience; the authors frame the release as support for future research on scalable visual-tactile learning. Teams training robot policies for contact-rich tasks have the most direct reason to watch it, since the reported gain is in real-world manipulation success.

How to use it

The dataset is not yet available to use. The authors say they will publicly release TouchScale, including all synchronized visual-tactile recordings and reconstructed object models. No release date, repository or download link for TouchScale is given. The paper describes three uses: training for zero-shot contact prediction, pretraining a visual encoder for action recognition, and visual-tactile mid-training of a robot policy.

How solid is it

The figures come from the authors' own abstract, so they are self-reported results. The success-rate numbers are absolute averages over four real-world tasks (22.5% to 57.5%), not relative gains. The scaling claim is worded as an overall upward trend, not a steady gain at every step, and the conclusion is phrased as "suggest". The abstract names no authors or institutions.

Risks and caveats

The number of real-world trials behind the 22.5% and 57.5% success rates is not given, so the size of the evidence for the robot result cannot be judged from the abstract. The robot policy, the robot platform and the four manipulation tasks are not named, nor is the tactile sensor or wearable hardware. The abstract does not say which prior visual-tactile datasets were compared or what the "prior tactile data" baseline is. The three action recognition benchmarks are not named and no accuracy values are given. The release is only promised.

“These results suggest that human visual-tactile data collected at scale with consistent sensing benefits both perception and robot manipulation.”

— Abstract, TouchScale paper