# Cross-Embodiment Transfer Should Be Done at the Experience Layer

Cross-embodiment transfer should be done at the experience layer

Full article: https://haiguangboy.com/posts/pegasus-editable-experience

## Core Method

↳ ① A five-stage pipeline: task graph → hierarchical affordance latent variables → cross-embodiment graph translation → physical constraint verification → experience rendering. Cross-embodiment transfer thus becomes a graph transformation problem, not an appearance generation problem.
↳ ② The invariant chosen is affordance, not object identity or pose. The authors first rule out three candidates: robot pose depends on the embodiment, human skeletons are likewise morphology-dependent, and object identity fails to generalize to new instances.
↳ ③ The generator only handles diversity; feasibility is left to an independent verifier. When inverse kinematics, collisions, or joint limits are violated, the verifier produces structured error signals fed back, with up to five retry rounds.

## Key Results

Five data sources are compared: graph-conditioned generation improves by 25.8% over human video features and by 13.5% over text-prompt-only generation. The authors attribute the gain to graph structure, not the video generation model itself. Ablating graph components consistently shows that removing the constraint graph causes the largest drop in executability.

## Related Work Comparison

- Same line · [LAWM: Why Action Labels Become a Burden](https://haiguangboy.com/posts/latent_action_pretraining_through_world_modeling) `latent_action_pretraining_through_world_modeling_2026_07`: LAWM: Why action labels become a burden
- Same line · [It's Not "Whether to Have a World Model," It's "Whether What It Outputs Has Structure"](https://haiguangboy.com/posts/robointer15_a_holistic_intermediate_representation_suite_for_embodied_world_mode) `robointer15_a_holistic_intermediate_representation_suite_for_embodied_world_mode_2026_07`: It's not "whether to have a world model," it's "whether what it outputs has structure"
- Same line · [Switching Logic Shouldn't Be Learned by the Policy](https://haiguangboy.com/posts/dr-lfd) `decompose_and_reorganize_planning_with_primitives_and_visuomotor_policies_learne_2026_08`: Graph translation is decomposed into three operator cascades: affordance graph → constraint graph → planning graph. Validates the positioning of foundation models: VLM semantics are only useful when converted into verifiable action structures.
- Same line · [Professor Li Hongyang's Whole-Body Intelligence Surpasses GR's Whole-Body Control](https://haiguangboy.com/posts/archon-whole-body-intelligence) `archon_blog_whole_body_intelligence_cn_20260712_2026_07`: Key design: choosing affordance rather than object identity or pose as the cross-embodiment invariant, organized in three layers. Validates evaluation criteria: don't look at single demonstrations, look at the speed of capability improvement on new scenes, new hardware, and new tasks.
- Same line · [Liang Wenfeng's Four-Hour Investor Meeting Transcript](https://haiguangboy.com/posts/liangwenfeng-world-model) `wx_elsewhere别处发生_20260722_2026_07`: Five-stage pipeline: task graph → hierarchical affordance latent variables → cross-embodiment graph translation → physical constraint verification → experience rendering. Validates ★The divergence may stem from differing referents: Liang Wenfeng's parallel of "3D/video generation/world models" falls entirely into the "renderer" category, while robotics needs a "simulator".
- Same line · [Predictive Features Shouldn't Be Fed Directly to Diffusion Models](https://haiguangboy.com/posts/leapbot-wa) `leapbot_wa_world_anchor_action_models_via_predictive_latent_alignments_2026_07`: Five-stage pipeline: task graph → hierarchical affordance latent variables → cross-embodiment graph translation → physical constraint verification → experience rendering. Validates Key design: the world model only observes intent, not actions—deliberately depriving it of low-level motor commands to prevent collapsing into a control shortcut.

## Limitations

This paper's credibility should be discounted for more than one reason. The generated trajectories were never executed on real hardware—the so-called physical validity comes from offline checks by the authors' own verifier, which they admit only handles kinematics, not full contact dynamics. The core metrics are custom-defined, with task correctness judged by a VLM, meaning the evaluator and generator share the same origin. In the cross-embodiment main table, the grasp stability column shows 100% across all five embodiments, while other columns in the same table exhibit normal variation. The generation stage of the entire pipeline relies on a commercial video generation API, with reproducibility and cost left undiscussed. Additionally, the main text contains language addressed to reviewers, which is uncommon in formal papers.

## Author's Assessment (Not Paper Content, Cross-Paper Synthesis)

Setting aside the numbers, there is one genuine divergence worth noting. Another work argues: rather than making the policy generalize to unseen situations, first use primitives to push the world back within the policy's distribution—what changes is not the model's generalization ability, but the state it is in when invoked. This paper takes the opposite stance, advocating for affordance composition to let the model infer how to operate unseen objects on its own.

Both paths answer the same question, but point to opposite ends. One keeps uncertainty outside the policy; the other demands compositional reasoning from the model. The former is more controllable in engineering, while the latter has higher ceiling but is harder to verify—and this paper precisely lacks real-robot execution to support the latter.