# LAWM: Why Action Labels Become a Burden

LAWM: Why Action Labels Become a Burden

Full article: https://haiguangboy.com/posts/latent_action_pretraining_through_world_modeling

## Core Method

LAWM's measured results are the opposite.

## Key Results

With the same batch of data, one group used BridgeData's real action labels for supervised pretraining, while the other received no labels at all and was only tasked with predicting the next video frame. The latter won. And within the winning group, there was an even more extreme variant: using only human videos—with no robot-executable actions—the real-task success rate jumped from 84% to 94%.

Where's the problem?

The authors' explanation: pretraining with action labels forces the model to learn something very specific—**given this robot, this trajectory, and this task setup, what values to output when seeing this image**. The model spends a large portion of its capacity fitting this particular mapping. Predicting the next frame, however, pushes the model to learn something else: **push it and it falls, let go and it drops, this angle is out of reach**. The former is tied to a specific embodiment; the latter is the physics of the world itself, which holds even when switching robots.

The more accurate the labels, the tighter the binding.

This explains why the other two numbers appear. First: a 7M-parameter small model with this pretraining matched a 7B-parameter counterpart—if it's learning general dynamics rather than a specific mapping, you don't need that much capacity to memorize. Second: 20 demonstrations with pretraining (77%) beat 50 demonstrations trained from scratch (70.5%)—the role of expert data is redefined; it no longer "teaches the model what actions are," but only aligns a model that already understands physics to a specific action space, which doesn't require as many samples.

## Related Work Comparison

- Same line · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Counterintuitive core finding: world model pretraining without action labels matches or exceeds supervised pretraining with labels, validating that joint co-training of heterogeneous human-robot data is structurally suboptimal
- Same line · [T-Rex: Why Tactile Sensing Needs Separate Modeling](https://haiguangboy.com/posts/t_rex_tactile_reactive_dexterous_manipulation) `t_rex_tactile_reactive_dexterous_manipulation_2026_07`: T-Rex: Why tactile sensing needs separate modeling
- Same line · [wx_界面新闻_20260605](https://haiguangboy.com/posts/wx_界面新闻_20260605) `wx_界面新闻_20260605_2026_06`: Comparing LAPA/UniVLA/villa-X: small models (7M-parameter scale) match or surpass large models (7B-parameter scale) of similar methods, validating the architectural bet: unified networks beat modular stitching—WUM opposes VLA's semantic layer-by-layer transmission
- Same line · [π0.5 trembles in place when failing to grab a spoon, but Orca goes further with physics intuition from watching videos](https://haiguangboy.com/posts/orca) `orca_2026_07`: The key to world models is readable states
- Same line · [No need to imagine the future at inference, robots still hit 91.8%! Fast-WAM debunks WAM's core assumption](https://haiguangboy.com/posts/fast-wam) `fast-wam_2026_07`: Training video objectives matter more than imagining the future at test time
- Same line · [Robots begin to 'stand in the light': Lingchu Intelligence enters optical module production lines](https://haiguangboy.com/posts/lingchu-optical) `wx_星河频率_20260718_2026_07`: This paper is a concrete engineering implementation of the 'massive Non-Action Data pretraining + small Action Data alignment' judgment, validating the 'native human data' pyramid claim: pretraining primarily on human data, with real-robot data only for post-training adaptation

## Limitations

The authors list three unresolved issues themselves: pixel reconstruction as a training objective rewards "looking like the image" rather than "task-relevant dynamics," so the model may learn background shortcuts; only a single viewpoint is used, without proprioception or gripper state; most critically, only 10% of the dataset is currently used, and success rates have already plateaued—whether adding more data can break through, the paper explicitly says it doesn't know. Real validation is also limited to five single-arm tasks.

## Author's Assessment (Not from the paper, cross-paper synthesis)

💡 This clue has repeatedly appeared in the library

The same judgment, different teams hit it in completely different ways: T-Rex took a three-stage recipe (29,000 hours of human video as a base, 100 hours of tactile data for grounding); Xinghitu Fast-WAM found that world model capabilities mainly come from representations learned during training, and inference-time frame prediction can be cut entirely; other work in the library has shown that "human video + 1 robot demonstration" can exceed pure robot data baselines.

Notably, these efforts are unaware of each other and take different routes, but all point to the same thing: **what robots truly lack may not be action labels, but methods to convert much larger-scale 'non-action data' into executable capabilities.**

But don't rush to treat it as settled. Also in the library, LAWM's "small model unified learning" result supports the WUM bet that "unified networks beat modular stitching," while T-Rex's conclusion is the opposite—it insists tactile sensing must go through an independent pathway, opposing forced fusion. On the same question of "whether one model should handle everything," the two papers stand on different sides. The essence of the disagreement may not be who's right, but: what should be unified (the world's physical laws) and what shouldn't (sensory signals whose timescales differ by orders of magnitude).