# KODA: Turning Human Demonstrations into Humanoid Whole-Body Data

KODA: Turning Human Demonstrations into Humanoid Whole-Body Data

Full article: https://haiguangboy.com/posts/koda

## Core Method

KODA converts human demonstrations into reusable "whole-body trajectories + robot viewpoint + task instructions." The key is to let a single generated trajectory simultaneously determine how the robot camera moves and what the deployed controller must track, and then generate training supervision from that.

↳ ① Generate whole-body motion from sparse references
Using camera and two-hand references, it retargets the hand-object geometry, then optimizes a whole-body trajectory via trackers and sampled commands in simulation, and finally refines the arms to reduce wrist tracking error.

↳ ② Make the robot view follow the generated motion
The whole-body trajectory determines the robot camera pose; the scene is reprojected, human arms are removed and the robot is rendered, and the remaining holes are filled in. The synthesized view corresponds to this robot motion, and inpainting only handles appearance without altering action labels.

↳ ③ Swap controllers, keep the recording, rebuild supervision
When switching between SONIC or HEFT, the canonical trajectory and synthesized views are kept, the commands for that controller are re-optimized, and the actual simulation states are recorded. The resulting state-action supervision is co-trained with the corresponding robot demonstrations for each GR00T N1.7 policy.

## Key Results

Across four G1 mobile manipulation tasks, each task uses a fixed 50 robot demonstrations plus 100 converted human demonstrations. Under variation conditions not covered by robot data but covered by human data, the average success rate across the four tasks:
· SONIC：3.75%→48.75%。
· HEFT：5.00%→51.25%。
Under both controllers, the average Robot-OOD success rate shows an upward trend within the tested human data budgets.

## Comparison with Related Lines

- Opposite line · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Real-robot training uses robot and converted human data co-training contradicts joint co-training of heterogeneous human-robot data is a structurally suboptimal approach
- Opposite line · [OpenWAM: The right answer changes with scale](https://haiguangboy.com/posts/openwam_an_open_modular_exploration_towards_systematic_world_action_model_pretra) `openwam_an_open_modular_exploration_towards_systematic_world_action_model_pretra_2026_09`: OpenWAM: The right answer changes with scale
- Opposite line · [Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-design](https://haiguangboy.com/posts/transformer-transformer-codesign) `transformer-transformer_blog_transformer_transformer_a_unified_20260807_2026_08`: Cross-controller reuse is still limited to the same embodiment and requires regenerating supervision contradicts the same DiT switches between generator, designer, and cross-embodiment controllers by changing condition masks
- Opposite line · [nvidia_gr00t_n1_7_egoscale_blog_20260417](https://haiguangboy.com/posts/nvidia_gr00t_n1_7_egoscale_blog_20260417) `nvidia_gr00t_n1_7_egoscale_blog_20260417_2026_08`: Data scaling trends are insufficient to establish a scaling law contradicts NVIDIA GR00T N1.7 bets on the "scaling human first-person video" route: 20,000+ hours of EgoScale pretraining, with the company claiming to have found the "first dexterity scaling law"—from 1,000 to 20,000 hours of data, average task completion more than doubles, without relying on massive teleoperation
- Opposite line · [Fusing RGB, 3D geometry, and object semantics into a single representation—this is the fundamental reason it is more accurate than π0.5 and Fast-WAM](https://haiguangboy.com/posts/flex_pi_a_multi_stream_world_action_model_with_compute_flexibility) `flex_pi_a_multi_stream_world_action_model_with_compute_flexibility_2026_08`: FLEX-π: a more comprehensive joint representation
- Opposite line · [Teleoperation data noise must enter the loss function](https://haiguangboy.com/posts/dexora) `dexora_open_source_vla_for_high_dof_bimanual_dexterity_2026_08`: Cross-controller reuse is still limited to the same embodiment and requires regenerating supervision contradicts the cross-embodiment premise: high-DoF policies treat low-DoF embodiments as subspaces, and dimensionality reduction is easier than dimensionality increase

## Boundaries

⚠️ Costs and boundaries
All 400 human demonstrations in this work include phone-measured depth and cannot be equated with arbitrary pure-RGB video. Cross-controller validation is still on the same G1, requiring regenerated supervision and separately trained policies. Each configuration is trained only once, with 20 real-robot trials per task per condition; the scaling trend is not yet sufficient to establish a scaling law.

## Author's Assessment (not paper content; cross-paper synthesis)

💡 Engineering insights
Supervision mismatch can also cause negative transfer: after SONIC adds converted data without command optimization, the success rate on already-covered conditions is 48.75%, lower than the 62.5% with robot data only.
Compared with WholeBodyWAM's cross-controller results, data reusability and policy adaptation-free operation are two separate problems. KODA places reuse in motion and observation records, leaving interface differences to supervision generation. What to watch next: whether task gains can be preserved and repeated conversion costs reduced under more controllers.