# MT-WAM: How Future Supervision Serves Action

MT-WAM: How Future Supervision Serves Action

Full article: https://haiguangboy.com/posts/mt-wam

## Core Method

↳ Retains joint video-action training, adds CoTracker3's future 2D point trajectories and frozen DINOv2's future visual feature supervision; duplicates the end of the video backbone to form a motion stream and a visual feature stream, sharing the front portion.

↳ Action directly reads the current frame and motion tokens; the visual feature stream is not directly read by action, and only shapes the shared backbone through training. What to learn and what to let action see are split into two design questions.

↳ At deployment, future video and auxiliary prediction heads are skipped, but the motion branch is retained. Each replanning runs the backbone and motion branch once, caching for multi-step action denoising reuse; "one forward pass" does not equal one-step action generation.

## Key Results

Keeping the parameter count and objective unchanged, only letting action additionally read the visual feature stream drops LIBERO-Plus success rate from 73.7% to 70.9%; removing the visual feature stream entirely drops it further to 68.8%. It is useful for training, yet not suitable for direct action reading in this setting.

## Comparison with Related Lines

- Opposite line · [OpenWAM: The Correct Answer Changes with Scale](https://haiguangboy.com/posts/openwam_an_open_modular_exploration_towards_systematic_world_action_model_pretra) `openwam_an_open_modular_exploration_towards_systematic_world_action_model_pretra_2026_09`: OpenWAM: The Correct Answer Changes with Scale
- Opposite line · [π0.5 Trembles in Place When It Fails to Grab a Spoon, While Orca Advances Further with Physical Intuition Gained from Watching Videos](https://haiguangboy.com/posts/orca) `orca_2026_07`: The key to a world model is a readable state
- Opposite line · [The More Accurate the Future Prediction, the More Sluggish the Robot](https://haiguangboy.com/posts/omega-0) `omega_0_a_latent_predictive_world_action_model_for_concurrent_humanoid_loco_mani_2026_08`: Duplicating the end of the video backbone to form dual-stream branches, rather than only adding output heads contradicts core design: future prediction is not treated as a video generation target, but as a compact prediction signal unrelated to reconstruction; the action branch does not need pixel-level generation or test-time video-to-action
- Opposite line · [Lower Prediction Error, Yet the Representation Has Already Collapsed](https://haiguangboy.com/posts/slim-05b) `slim_05b_learning_action_grounded_predictive_latents_for_robot_manipulation_2026_08`: Retains FAST-WAM's direct action interface, adds two types of future supervision contradicts Stage 2 removing the entire future latent variable, keeping only the predictive slot shaped by Stage 1
- Same line · [Agile-WAM: Lightweight Visuotactile Joint Prediction](https://haiguangboy.com/posts/agile-wam) `agile_wam_an_agile_tactile_world_action_model_for_contact_rich_robot_control_2026_09`: Retains FAST-WAM's direct action interface, adds two types of future supervision validates the authors' claim: whichever modalities are input should also have their futures learned
- Same line · [JEPA-WAM: An Innovative Method Makes World Models Better Understand State Changes](https://haiguangboy.com/posts/jepa-wam) `jepa_wam_learning_vision_language_action_policies_with_joint_embedding_world_modeling_2026_08`: Authors' judgment: after omitting future generation, priority should be given to improving the current single forward-pass representation validates core claim: transition supervision must directly shape the backbone that generates actions; neither external attachment nor serving as context works

## Boundaries

⚠️ Engineering cost and boundaries
On the same RTX PRO 6000, latency increases by only 5.79%, while FLOPs increase by 37.63%. 2D point trajectories do not represent mastery of 3D contact dynamics; existing ablations also do not fully disentangle the respective contributions of extra capacity and auxiliary supervision.

## Authors' Judgment (not paper content, cross-paper synthesis view)

💡 Placed in the context of related work
TacWAM uses future touch for supervision and finds in nested ablations that gradually opening up information interaction harms control; this echoes MT-WAM's asymmetric division of labor.

But OpenWAM finds that cutting off action's access to world features drops about 5 points. The two do not directly negate each other: the world stream and the auxiliary visual feature stream are not the same kind of signal, and MT-WAM also retains action's reading of the motion stream. What truly should be verified item by item in engineering is: which objective improves representation, and which information pathway improves control.