# N0-Foundation: Ushering in a New Era of Tactile Intelligence

N0-Foundation: Ushering in a New Era of Tactile Intelligence

Full article: https://haiguangboy.com/posts/mathcaln_0_foundation_towards_the_age_of_tactile_intelligence

## Core Approach

Hardware + data + representation + evaluation fully integrated; the authors call it the first step toward the "age of tactile intelligence"
↳ Tactile sensors are vision-based: the elastic sensing layer deforms under force, and a camera captures texture displacement images. Most works directly use this image as the tactile representation; this work goes further: it learns an inverse mapping to recover a three-axis force field (2 shear forces + 1 pressure) at each spatial position, using the force field itself as a unified representation—physically grounded, cross-sensor unified (calibrated sensors can map into the same space), and spatially semantically rich (preserving the full distribution rather than summarizing to a single number)
↳ This representation remains consistent across five contact modes—press, twist, grasp, lift, and hold—that differ greatly, because the representation space is defined by physics, not by device or behavior
↳ The accompanying handheld device N0-TacUMI contributes 57% of trajectories, plus teleoperation across 5 robot platforms, aggregating into NeoData with 30,000+ hours and 1.4 million episodes

## Key Results

· On 10 real-world tasks, feeding the force field to an action expert outperforms no tactile input and raw image concatenation, with progressive scores rising from 38.1% to 47.5%
· Gains concentrate on force field magnitude accuracy, not contact localization accuracy—this aspect still has room for improvement
· Across 12 simulated tasks, sustained bimanual contact tasks are nearly unsolvable (best 18%-25%); Fast-WAM tops real-world tests but fails all 12 simulated tasks

## Comparison of Related Approaches

- Opposing approach · [PRISM: Can Scale Replace Explicit Modeling?](https://haiguangboy.com/posts/prism_precision_and_contact_rich_real_world_industrial_skill_dataset_with_multim) `prism_precision_and_contact_rich_real_world_industrial_skill_dataset_with_multim_2026_08`: PRISM: Can Scale Replace Explicit Modeling?
- Opposing approach · [Tactile prediction should only be used for training, not fed to actions](https://haiguangboy.com/posts/tacwam) `tacwam_anchor_guided_world_action_model_with_mechanics_aware_tactile_prediction_2026_08`: The authors argue that the bottleneck of existing policies on fine-grained contact-rich tasks stems fundamentally from pure vision plus proprioception failing to expose contact states or provide signals needed for mid-action correction—not from insufficient policy capacity, but from a missing modality in perception itself contradicts core principle: future tactile input serves only as training-time supervision, not deployment-time action input
- Same approach · [JEPA-WAM: Innovative Methods Make World Models Better Understand State Changes](https://haiguangboy.com/posts/jepa-wam) `jepa_wam_learning_vision_language_action_policies_with_joint_embedding_world_modeling_2026_08`: Tactile integration ablation (fixed π0.5 backbone): no tactile < raw tactile image concatenation < raw tactile image fed to action expert < NeoForce force field fed to action expert, with average progressive scores rising from 38.1% to 47.5% validates core claim: transfer supervision must directly shape the backbone generating actions; external add-ons or context use are insufficient
- Same approach · [S1: The GPT-3 Emergence Moment for Robots](https://haiguangboy.com/posts/skild_blog_introducing_s1_in_context_learning_20260817) `skild_blog_introducing_s1_in_context_learning_20260817_2026_08`: S1: The GPT-3 Emergence Moment for Robots
- Same approach · [T-Rex: Why Tactile Sensing Needs Separate Modeling](https://haiguangboy.com/posts/t_rex_tactile_reactive_dexterous_manipulation) `t_rex_tactile_reactive_dexterous_manipulation_2026_07`: T-Rex: Why Tactile Sensing Needs Separate Modeling
- Same approach · [Glove tactile readings are mixed with hand pose artifacts](https://haiguangboy.com/posts/tactile-glove-pose-artifacts) `pose_aware_modeling_to_mitigate_pose_related_artifacts_in_tactile_gloves_2026_07`: Tactile sensors are vision-based (camera-based visuo-tactile): the elastic sensing layer deforms under force, an internal camera captures texture displacement, and an inverse mapping is learned to recover the force field from images, rather than directly using raw images as tactile representations validates again that 'direct concatenation performs worse': this time explained by the model learning the wrong mapping direction

## Limitations

· Bimanual coordinated contact tasks still lag significantly; tactile sensing is necessary but not sufficient
· Only validated with parallel-jaw visuo-tactile fingers; dexterous hands and non-camera-based sensors are future directions
· Trajectories collected by the handheld device require inverse kinematics screening to confirm executability; not all collected data is usable

## Author's Assessment (not from the paper; synthesized across papers)

The sharpest contrast is with TacWAM: TacWAM claims future tactile input should only serve as training-time supervision and not enter deployment-time action input, while this paper's ablation shows feeding the force field to deployment-time action experts yields the greatest gains—the two give opposite answers; however, TacWAM's diagnosis that 'tactile input has structural temporal ambiguity and single frames are insufficient' is indirectly acknowledged here, as the force field representation specifically aggregates temporal context within segments to address it. It aligns with T-Rex on the same path: tactile sensing requires dedicated asynchronous processing, and naive concatenation is ineffective; this paper refrains from directly concatenating raw tactile images into the policy, serving as another independent confirmation of the same judgment; T-Rex achieves 65% success on 12 real-world tasks, 30 percentage points above the strongest baseline, while this paper is larger in scale, with 10 tasks and 30,000+ hours of data. It continues the same trajectory as N0-VTLA: both use the same NeoReal benchmark; N0-VTLA previously scored 47.2% against π0.5's 29.4%, and this paper expands to 10 tasks, raising π0.5 to 26.5%—but it also inherits N0-VTLA's same limitation: evaluations are primarily on self-built benchmarks, heavily relying on proprietary sensors and self-constructed data pipelines.