# π0.5 freezes and trembles in place when it fails to grab a spoon, but Orca pushes physical intuition further by learning from watching videos The key to a world model is a readable state Full article: https://haiguangboy.com/posts/orca ## Core Method # π0.5 freezes and trembles in place when it fails to grab a spoon, but Orca pushes physical intuition further by learning from watching videos ## Key Results ## One-Sentence Takeaway The key to a world model is a readable state ## What Problem Does the Paper Solve? When robot manipulation tasks fail, most policies either retry endlessly in place or get completely stuck with no way to move forward—this is a more critical issue than "success rate" in real deployment: tasks can never succeed on the first try every time, and whether the system can correct course after failure and keep going is what truly determines usability. ## Core Method ↳ ① Instead of predicting tokens/frames/actions, learn a unified world latent space: unconscious learning (learning natural state transitions from continuous video) + conscious learning (learning purposeful state transitions via language events + VQA) ↳ ② The backbone training is completely frozen, only lightweight downstream readout modules are trained—text direct output, image via diffusion model decoding, action via Action Expert—validating whether the latent space itself is truly useful ↳ ③ Ablations confirm all three training objectives are indispensable: using only observation objectives scores 29.3, using all three reaches 48.0; removing the observation objective drops action generation scores from 32.4 to 10.2—dense physical dynamics information in video is exactly what action generation relies on most ## Experimental Results · Action generation: despite never using action labels during pretraining, Qwen3.5 breaks through from 0% success rate in OOD settings and matches the strong baseline π0.5 pretrained on large-scale robot data · Grasp failure recovery: measured by FNS (failure stage number) and DRR (retreat recovery ratio), Orca advances to later stages even when it ultimately fails; in contrast, π0.5 remains unstable after repeated failures on the same task · Image prediction: at 0.8B scale it loses to all dedicated image generation baselines, but at 4B scale it surpasses the strongest among them—the same method, with only model scale changed, goes from last place to first ## What This Paper Is Not It is not a complete embodied world model. The authors openly admit in the Limitations section: using a frozen pretrained vision encoder as the supervision target is a design compromise to simplify training, not an ideal solution—a truly general world foundation model should learn a unified space directly from multi-source signals; moreover, the 4B/0.8B model scales already show trade-offs among language/image/action capabilities, and only one-tenth of the 125K hours of video has been used so far. ## Implications for Robotics / World Models / VLA The finding that Orca lets Qwen3.5 break through from 0% success rate and match π0.5 pretrained on large-scale robot data is judged in cross-library relational metabolism to strongly corroborate several other papers: Fast-WAM achieves near-SOTA without embodied pretraining, Ψ0 surpasses a 10x-data baseline by over 40% with only a small amount of data, and General Flow achieves zero-shot high success rate transfer via human video pretraining. Several papers with completely different technical routes are converging on the same direction: what is truly scarce is not the quantity of robot action data, but whether a high-quality world representation can be learned first. ## Key Pages of the Paper ![paper page](https://static.haiguangboy.com/papers/orca/page_01.webp) ![key figure](https://static.haiguangboy.com/papers/orca/key_figure.webp) ## Comparison of Related Approaches - Same route · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Action generation comparison: letting Qwen3.5 break through from 0% success rate and match π0.5 pretrained on large-scale robot data validates that joint co-training on heterogeneous human-robot data is a structurally suboptimal approach - Same route · [No need to imagine the future during inference, the robot still reaches 91.8%! Fast-WAM debunks WAM's core assumption](https://haiguangboy.com/posts/fast-wam) `fast-wam_2026_07`: Both show that the value of a world model lies not only in generating the future during inference, but in shaping transferable world representations during training. ## Boundaries ## Author's Judgment (not paper content, cross-paper synthesis) The finding that Orca lets Qwen3.5 break through from 0% success rate and match π0.5 pretrained on large-scale robot data is judged in cross-library relational metabolism to strongly corroborate several other papers: Fast-WAM achieves near-SOTA without embodied pretraining, Ψ0 surpasses a 10x-data baseline by over 40% with only a small amount of data, and General Flow achieves zero-shot high success rate transfer via human video pretraining. Several papers with completely different technical routes are converging on the same direction: what is truly scarce is not the quantity of robot action data, but whether a high-quality world representation can be learned first.