# Robot failures may not be about the policy being big enough

Robot failures may not be about the policy being big enough

Full article: https://haiguangboy.com/posts/transformer-transformer-codesign

## Core method

↳ ① RoboTokens compress links, joints, motors, states, actions, and target end-effector poses into unified continuous tokens. The project page reports that 11 Menagerie robots, spanning 0.65–67.5 kg and 6–35 actuated joints, each require only 28–101 tokens, 27–110 times fewer than MJCF text.

↳ ② The same DiT switches roles by changing the conditioning mask: without a fixed body, it acts as a robot generator; given a body and target motion, it becomes a cross-embodiment controller; given target motion, diffused body, and dynamics tokens, it serves as a task designer.

↳ ③ Dynamics Self-Guidance lets the model first predict the full state-action trajectory for candidate bodies, then backpropagates gradients from the user reward function into the body tokens. It does not train a separate critic for new rewards or rely on differentiable simulators, but uses the model's own dynamics predictions for guidance.

## Key results

The real ALOHA2 cloth-flinging demo is most convincing: the model-generated design lengthens the links and mounts the arms inverted at the rear of the workspace, turning overhead flinging into a more efficient underhand swing. The project page reports a 73% reduction in real-robot tracking error, a 30% reduction in peak joint velocity, and cloth that the original structure could not open is now unfolded.

Speed is also aggressive: CMA-ES requires generation-by-generation rollouts in MuJoCo; this method samples in parallel on GPU and predicts an entire episode at once. The project page reports that multi-trajectory bimanual optimization drops from over 3 hours to under 1 minute.

## Comparison with related approaches

- Opposite route · [wx_界面新闻_20260605](https://haiguangboy.com/posts/wx_界面新闻_20260605) `wx_界面新闻_20260605_2026_06`: claims task failures should not be default-attributed to the policy; body-controller-policy should enter the same optimization problem contradicts route bet: build the brain, not the body—betting on foundation model company identity, rejecting the locomotion capability track
- Same route · [1 day, 1000 tasks, relying on inductive biases](https://haiguangboy.com/posts/mt3-thousand-tasks) `learning_a_thousand_tasks_in_a_day_2026_08`: design space lacks complex meshes, scenes, and contact targets, leaving key gaps toward assembly and dexterous contact validates perception dependence: vision-only, single camera, no tactile sensing, reliant on accurate segmentation
- Same route · [RL should not start from scratch, it should be post-training](https://haiguangboy.com/posts/rl-100) `rl_100_performant_robotic_manipulation_with_real_world_reinforcement_learning_2026_08`: Dynamics Self-Guidance: backpropagating reward gradients from predicted full dynamics into body tokens validates modeling denoising as a two-level MDP, where K denoising steps share the same environment-level advantage, addressing weak credit assignment
- Same route · [latepost_xuhuazhe_202603](https://haiguangboy.com/posts/latepost_xuhuazhe_202603) `latepost_xuhuazhe_202603_2026_03`: problem definition: given end-effector target motion and reward function, jointly generate full body and controller validates route bet: behavior/action parts must be a unified model, opposing modular stitching
- Same route · [Switching logic should not be learned by the policy](https://haiguangboy.com/posts/dr-lfd) `decompose_and_reorganize_planning_with_primitives_and_visuomotor_policies_learne_2026_08`: claims task failures should not be default-attributed to the policy; body-controller-policy should enter the same optimization problem validates problem diagnosis: end-to-end policies are forced to learn both "how to do" and "when to switch," with data needs exploding combinatorially
- Same route · [The bottleneck in robot RL is sampling, not algorithms](https://haiguangboy.com/posts/pi-rl-chelsea-finn-talk) `x_manual_pi_rl_20260726_20260727_2026_07`: "seconds vs. hours" holds only in the amortized inference stage; new design spaces still incur high expert data costs validates engineering reading: the difference between robot RL and LLM post-training is the magnitude of sampling cost, not a contest of algorithmic superiority

## Boundaries

This is not an open-world VLA. Inputs are still preset end-effector trajectories and hand-written rewards, not automatic task understanding from language, images, or object states.

Nor is it a reliable final evaluator. Design quality depends on the same model for cross-embodiment control validation, yet its correlation coefficient with per-design RL oracles is only 0.53. Evidence also comes from self-reported project page results, with real-robot demos limited to one ALOHA variant and one cloth-flinging task.

## Author's judgment (not paper content, cross-paper synthesis)

This challenges the "build the brain, not the body" route. It argues that many failures are not policy-level issues, but stem from body-controller-task distributions not being co-designed. For general-purpose robot companies, this may be too heavy; but for fixed-station, fixed-task-distribution industrial systems, first adjusting mounting angles, links, tool geometry, and actuators, then doing policy post-training, may be closer to the right answer than scaling up the policy model.