# 50 demos hit 100%, others with 1000 only get 45%—the difference is it removed "when to switch tactics" from the strategy

Switching logic shouldn't be learned by the policy

Full article: https://haiguangboy.com/posts/dr-lfd

## Core method

↳ ① Classify by contact complexity into three types: pick/place uses equivariant networks to learn open-loop primitives; contact-dense tasks like wiping, flipping, folding use visual motor policies predicting key poses at both ends; no-contact tasks go directly to planned trajectories.
↳ ② VLM only does semantic grounding, not deciding split points. Where to split is determined by contact distance thresholds—between two contact change events is one segment.
↳ ③ Execution isn't a one-shot open-loop computation. The global skeleton is computed once, but before each action it re-perceives, solves local planning for that subgoal, and after execution verifies with contact detectors; if expected contact isn't achieved, it re-solves the same step.

## Key results

· Simulated peg-in-hole: 50 demos, 100% in-distribution, ACT 44%, DP 54%
· Dual-arm tasks with 100 demos hit 70% and 90%, while baselines with 1000 only got 45% and 70%
· Real-world out-of-distribution tests, diffusion policy dropped to 30%
· Cost is speed: single completion takes 10.21 seconds, baselines 7.5–8.7 seconds

## Related approach comparison

- Opposite route · [latepost_xuhuazhe_202603](https://haiguangboy.com/posts/latepost_xuhuazhe_202603) `latepost_xuhuazhe_202603_2026_03`: Problem diagnosis: end-to-end policies are forced to learn both "how to do" and "when to switch," data needs explode combinatorially contradicts route bet: behavior/action parts must be a unified model, opposing modular assembly
- Opposite route · [wx_界面新闻_20260605](https://haiguangboy.com/posts/wx_界面新闻_20260605) `wx_界面新闻_20260605_2026_06`: Core claim: switching logic should go to the planner, not be learned by the policy—this is a structural rebuttal to "end-to-end scaling" contradicts architecture bet: unified networks beat modular assembly—WUM opposes VLA's semantic layer-by-layer transfer
- Opposite route · [Tactile as prediction target rather than observation input](https://haiguangboy.com/posts/n_0_vtla_scaling_vision_tactile_language_action_model_with_latent_tactile_tokens) `n_0_vtla_scaling_vision_tactile_language_action_model_with_latent_tactile_tokens_2026_07`: Tactile as prediction target rather than observation input
- Same route · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: Problem diagnosis: end-to-end policies are forced to learn both "how to do" and "when to switch," data needs explode combinatorially validates joint co-training of heterogeneous human-robot data is structurally suboptimal
- Same route · [Gemini_Robotics_2_Safety](https://haiguangboy.com/posts/gemini_robotics_2_safety) `gemini_robotics_2_safety_2026_07`: Premise: symbolic planning and visual motor policies are complementary parallel lines validates empirical finding: clear gap between models "understanding constraints" and "acting by constraints"
- Same route · [LAWM: Why action labels become a burden](https://haiguangboy.com/posts/latent_action_pretraining_through_world_modeling) `latent_action_pretraining_through_world_modeling_2026_07`: LAWM: Why action labels become a burden

## Boundaries

Inherits TAMP's old debts: goals still need manual specification; search complexity grows exponentially with symbol count, obstacles going from 0 to 3 raises planning time from 9 seconds to 145 seconds, with a standard deviation of 190 seconds. Success rates in constrained scenarios are notably lower than in-distribution. Object-centric primitives require calibrated depth sensors. The VLM step's output needs manual checking and relies on unambiguous step-by-step descriptions—which real tasks usually lack; the authors wrote them themselves. Open-set segmentation also has vocabulary limits, with one obstacle unrecognized in about half the trials.

## Author's judgment (not from the paper, cross-paper synthesis)

The most transferable part isn't the architecture, it's the mechanism: rather than making the policy generalize out-of-distribution, first push the world back into the policy's distribution. Primitives move objects to in-distribution contact states, key poses bring the arm to starting postures the policy can accept. What changes isn't the model's generalization ability, but the state it's invoked in.

Looking across, recent papers give different answers to the same problem. Some advocate unified models against modular assembly; some oppose horizontal cuts like "navigation module plus grasping module," favoring vertical cuts by abstraction level; this one says cutting is fine, but after cutting someone must be responsible for pushing states back into distribution.

One direct opposition worth noting: a tactile work's deployment contract is that once an action chunk is issued it must execute fully, no truncation or replanning; this paper is the opposite—before each action it re-perceives and locally re-solves. One bets predictions are accurate enough not to interrupt, the other bets the world will drift so every step needs correction. This divergence should show results within six months.