# Massive human videos go unused, and data from different robots can't be stitched together—this paper solves both problems at once

The issue with heterogeneous data isn't volume, it's inconsistent supervision formats

Full article: https://haiguangboy.com/posts/joyai-ra-dual-alignment

## Core method

Route data based on the supervision type it actually has, rather than forcing a unified format. Unlabeled video follows implicit alignment: train a shared latent action model across human/simulation/robot, using visual transfer to derive transfer-level dynamics supervision on its own. Trajectories with reliable labels follow explicit alignment: uniformly map into a 130-dimensional cross-embodiment action space, with end-effector actions represented as block-wise relative poses in the camera frame.

Architecturally, a VLM handles semantics, a frozen world model handles dynamics, and late fusion feeds a flow-matching action expert. Three-stage training: first pretrain the world model, then freeze it and jointly train the VLM and action expert, and finally post-train on the target robot. After deployment, there's also inner-outer loop reinforcement learning, with edge-side fast adaptation to the current task and center-side continuous improvement of the base model synced back.

## Key results

On real robots, seen tasks average 92.0, versus the VLA baseline at 74.0, with the gap widening as task difficulty increases. Ablations are stark: removing both alignments drops unseen task scores to 29.8; removing only implicit alignment drops unseen scores alone to 46.8, while seen scores hold relatively. Scaling human video to the full corpus raises both seen and unseen scores, with unseen task gains notably larger in early stages.

## Related route comparisons

- Opposite route · [LAWM: Why action labels become a burden](https://haiguangboy.com/posts/latent_action_pretraining_through_world_modeling) `latent_action_pretraining_through_world_modeling_2026_07`: LAWM: Why action labels become a burden
- Opposite route · [latepost_xuhuazhe_202603](https://haiguangboy.com/posts/latepost_xuhuazhe_202603) `latepost_xuhuazhe_202603_2026_03`: Architecture: VLM handles semantics, frozen LAC-WM handles dynamics, late fusion feeds flow-matching action expert contradicts route bet: behavior/action parts must be a unified model, opposing modular stitching
- Opposite route · [Robots 'standing in the light': Lingchu Intelligence enters optical module production lines](https://haiguangboy.com/posts/lingchu-optical) `wx_星河频率_20260718_2026_07`: Pretraining corpus: 53K+ hours human video, 11K+ hours simulation, 8K+ hours real robot data, covering dual-arm and single-arm embodiments contradicts 'native human data' pyramid claim: pretraining dominated by human data, real robot data only for post-training adaptation
- Same route · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Core claim: reframe 'not enough robot data' as 'inconsistent supervision formats across heterogeneous sources', routing by available supervision type rather than forcing unified format validates joint co-training of heterogeneous human-robot data is structurally suboptimal
- Same route · [Where should RL intervene](https://haiguangboy.com/posts/zprl) `beyond_action_residuals_real_world_robot_policy_steering_via_bottleneck_latent_r_2026_08`: Inner-outer loop RL: edge-side fast inner loop adapts to current task, center-side asynchronous outer loop continuously improves base model then syncs back validates problem reframing: key for RL post-training isn't just 'how much to change', but 'at which layer to intervene'
- Same route · [T-Rex: Why touch needs separate modeling](https://haiguangboy.com/posts/t_rex_tactile_reactive_dexterous_manipulation) `t_rex_tactile_reactive_dexterous_manipulation_2026_07`: T-Rex: Why touch needs separate modeling

## Boundaries

Real robot evaluation has only 20 seen and 10 unseen trials per task. Comparison includes just one VLA baseline, without a broader method family. Inner-outer loop RL is called a self-improvement system, but authors admit outer loop sync frequency is deliberately suppressed—high-frequency sync causes training instability, which remains unsolved.

## Author's judgment (not paper content, cross-paper synthesis)

The strongest point here is the problem reframing itself—changing 'not enough data' to 'inconsistent supervision formats'. This aligns with several other works' directions, especially the philosophy of 'massive non-action data pretraining + small action data alignment', and this paper offers a concrete engineering implementation.

But one area deserves deeper thought. This paper's ablations show removing only implicit alignment from unlabeled video drops unseen generalization the most—suggesting labeled data is more critical for precision. Yet another work's counterintuitive finding is that world model pretraining with no action labels at all can match or exceed supervised pretraining with labels. Both are empirical, but directions conflict. The likely difference is how each uses 'world model pretraining'—one uses it to shore up unseen generalization gaps, the other to fully replace labeled pretraining, so they're not actually asking the same question.