# Tactile prediction should only be used for training, not for action

Tactile prediction should only be used for training, not for action

Full article: https://haiguangboy.com/posts/tacwam

## Core Method

↳ ① Future tactile signals serve only as supervision targets during training, not as action inputs during deployment. The authors explicitly note that tactile prediction itself is not novel; their focus is on how future tactile data is represented and used internally during training.
↳ ② Attention masks enforce strict isolation: action tokens cannot read future visual and tactile predictions, and video tokens cannot read tactile futures. The authors clearly define this as an information isolation mechanism, not an argument for action-conditioned tactile dynamics.
↳ ③ Tactile encoding fuses three spatially aligned signals—tactile images, dense force fields, and mesh deformation flow—with bilateral resultant forces and torques as global mechanical supervision, predicting latent states rather than directly generating sensor streams.
↳ ④ Recent tactile history is not inserted as extra tokens but modulates the tactile expert via adaptive normalization—the same force magnitude could indicate stable contact, pressure buildup, slip recovery, or imminent crushing, and only history can disambiguate.

## Key Results

Across four contact-intensive tasks on a real robot with 20 trials each, the mean success rate was 75.0%, versus 37.5% for the strongest baseline, winning every task. The largest gains were in wiping a whiteboard and pen spinning, which rely most on sustained contact and bilateral dynamic reasoning.

The most valuable finding is the ablation curve: full version at 82.5%; removing tactile history drops to 55.0%; further allowing action streams to see tactile predictions drops to 37.5%; and permitting bidirectional interaction between visual and tactile futures drops to 7.5%.

## Related Work Comparison

- Opposite approach · [Tactile as prediction target rather than observation input](https://haiguangboy.com/posts/n_0_vtla_scaling_vision_tactile_language_action_model_with_latent_tactile_tokens) `n_0_vtla_scaling_vision_tactile_language_action_model_with_latent_tactile_tokens_2026_07`: Tactile as prediction target rather than observation input
- Same approach · [LAWM: Why action labels become a burden](https://haiguangboy.com/posts/latent_action_pretraining_through_world_modeling) `latent_action_pretraining_through_world_modeling_2026_07`: LAWM: Why action labels become a burden
- Same approach · [T-Rex: Why tactile needs separate modeling](https://haiguangboy.com/posts/t_rex_tactile_reactive_dexterous_manipulation) `t_rex_tactile_reactive_dexterous_manipulation_2026_07`: T-Rex: Why tactile needs separate modeling
- Same approach · [π0.5 trembles in place when grabbing a spoon, but Orca's physics intuition from watching videos goes further](https://haiguangboy.com/posts/orca) `orca_2026_07`: The key to world models is readable states
- Same approach · [It's not "whether to have a world model," but "whether what it outputs has structure"](https://haiguangboy.com/posts/robointer15_a_holistic_intermediate_representation_suite_for_embodied_world_mode) `robointer15_a_holistic_intermediate_representation_suite_for_embodied_world_mode_2026_07`: It's not "whether to have a world model," but "whether what it outputs has structure"
- Same approach · [Glove tactile readings are mixed with hand pose](https://haiguangboy.com/posts/tactile-glove-pose-artifacts) `pose_aware_modeling_to_mitigate_pose_related_artifacts_in_tactile_gloves_2026_07`: Very small scale: single robotic arm platform, four tasks, 300 demos per task, 20 trials per method; statistical foundation is thin: only 5 people per glove, p-values discretized at 0.03125; no zero-shot cross-glove transfer

## Limitations

Single-arm platform, four tasks, only 20 trials per method per task—each success or failure is 5 percentage points, and small gaps between tasks fall within the range of two trials. The strongest baseline lacks an official implementation and was reproduced by the authors themselves, yet it serves as the comparison for that 37.5-point improvement. Additionally, tactile prediction is not used in closed loop during deployment; all gains come from improved representations during training, not from "the robot adjusting in real time based on predicted contact."

## Author's Assessment (Not from the paper, cross-paper synthesis)

That ablation curve deserves special attention. The last two ablation levels do not reduce any available information; they only remove structural constraints on information flow, yet the loss exceeds that of the first level's information removal. The judgment that "auxiliary signals cannot be fed into the backbone without constraints" has previously only been supported by positive evidence that "adding constraints improves performance"; this is the first time we see a descending curve from progressively relaxing constraints.

There's also an interesting divergence within the same school. Another tactile work similarly advocates treating tactile as a prediction target rather than an observation input, but it feeds the predicted latent variables to the action expert to condition actions; this paper explicitly prevents actions from reading tactile futures. Both start from the same point but give opposite answers to "whether predicted outputs should be shown to actions." And this paper's ablation directly tests this, showing a 17.5-point drop.