# T-Rex: Why Tactile Sensing Needs Its Own Model

T-Rex: Why Tactile Sensing Needs Its Own Model

Full article: https://haiguangboy.com/posts/t_rex_tactile_reactive_dexterous_manipulation

## Core Approach

T-Rex's answer: likely not. Tactile sensing is not just another image.

## Key Results

Vision primarily handles "what the object is, where the goal is, and roughly what to do next"; tactile sensing, however, processes pressure, slip, deformation, and force changes at the moment of contact. The former relies on low-frequency semantic reasoning, while the latter demands high-frequency, local, almost reflexive closed-loop corrections. A model trained predominantly on visual data, even if it can accept tactile input at the interface, does not truly learn to use tactile sensing—new signals may be overwhelmed by visual features or even drag down the policy.

T-Rex therefore does not mechanically stuff tactile data into a vision backbone but builds a dedicated pathway for it:

① A slow visual-motion expert handles scene understanding, language instructions, and coarse-grained actions;
② A fast tactile expert bypasses the slow visual reasoning chain, using high-frequency contact feedback to refine fine-grained actions;
③ Within tactile processing, further separation occurs: dynamic forces use temporal encoding, while surface deformation uses spatial encoding, respectively answering "how forces change" and "where contact occurs."

This is not an isolated architectural preference. Another independent study in the knowledge base found that directly concatenating force/tactile signals into VLA state vectors degrades performance; other works also shift toward dedicated encoders and pathways. Multiple teams converge from different experiments on the same direction: the difficulty with tactile sensing is not data availability but that vision-centric representations and timescales struggle to absorb it directly and stably.

T-Rex's data recipe also reflects this division: first, 22,900 hours of human first-person video establishes visual and motor priors; then, about 100 hours of robot tactile data grounds it; finally, post-training targets specific skills. Not all data is mixed from day one; instead, different data separately address "understanding the world" and "aligning tactile control."

In real-world tests, T-Rex achieves an average success rate of 65% across 12 real tasks, surpassing the strongest baseline by over 30 percentage points, with ablations confirming the independent contribution of tactile encoding methods.

## Related Route Comparisons

- Same route · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Three-stage training recipe: large-scale human first-person pretraining → tactile grounding training → skill post-training validates that joint co-training of heterogeneous human-robot data is structurally suboptimal
- Opposite route · [wx_界面新闻_20260605](https://haiguangboy.com/posts/wx_界面新闻_20260605) `wx_界面新闻_20260605_2026_06`: Route judgment: tactile is a signal fundamentally different from vision, requiring dedicated asynchronous architectures; mechanical fusion is ineffective contradicts architectural bet: unified networks outperform modular concatenation—WUM opposes VLA's semantic layer-by-layer transfer
- Same route · [latepost_xuhuazhe_202603](https://haiguangboy.com/posts/latepost_xuhuazhe_202603) `latepost_xuhuazhe_202603_2026_03`: Boundary: single robot platform + lab-collected environment + self-reported evaluation, including one actively disclosed failure case validates boundary: single media interview self-report, no third-party verification, founder admits path uncertainty
- Same route · [sunday_blog_20260717](https://haiguangboy.com/posts/sunday_blog_20260717) `sunday_blog_20260717_2026_07`: Boundary: single robot platform + lab-collected environment + self-reported evaluation, including one actively disclosed failure case validates source boundary: self-reported, self-built evaluation, no third-party, single task family
- Same route · [π0.5抓勺子失败就原地哆嗦,Orca却靠刷视频悟出的物理直觉更进一步](https://haiguangboy.com/posts/orca) `orca_2026_07`: The key to world models is readable states
- Same route · [机器人开始「站在光里」：灵初智能切入光模块产线](https://haiguangboy.com/posts/lingchu-optical) `wx_星河频率_20260718_2026_07`: Route echo: three-stage recipe embodies the philosophy of 'massive non-action data pretraining + small action data alignment' validates 'native human data' pyramid claim: pretraining dominated by human data, real-robot data only for post-training adaptation

## Boundaries

It has not yet demonstrated generalization across different embodiments, homes, or production lines; experiments rely on a single platform and controlled environments, and the 65% figure is self-reported by the authors with no third-party replication.

## Author's Judgment (Non-paper content, cross-paper synthesis)

💡 True route judgment

A unified robot system does not mean all modalities must be crammed into the same encoder, using the same frequency and representation.

A more likely architecture: each physical signal first forms specialized neural pathways based on its own nature, then coordinates in action decisions or shared latent spaces. Robots can have a unified brain, but not necessarily only one sensory pathway.

On a deeper level, T-Rex's three-stage training echoes another recurring route in the knowledge base: massive human video first builds physical and motor priors, with small robot data for final alignment. What is truly scarce may not just be action labels, but how to convert larger-scale 'non-action data' into executable capabilities.